Exploring affective image representation with visual attention and aesthetic fusion

Jixiong Ma; Hao Zhang; Kangjian He; Dan Xu

doi:10.1117/12.2680098

27 June 2023 Exploring affective image representation with visual attention and aesthetic fusion

Jixiong Ma, Hao Zhang, Kangjian He, Dan Xu

Proceedings Volume 12705, Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022); 127051U (2023) https://doi.org/10.1117/12.2680098
Event: Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022), 2022, Nanjing, China

Abstract

Affective image analysis aims to understand the sentiment of different images. The challenge is to develop a discriminative representation that bridges the affective gap between low-level features and high-level emotions. Most existing studies bridge the gap by designing deep models carefully to learn global representations in one shot directly or identify image emotion by extracting features at different levels in the model. They ignore that both local regions of an image and relationships between them impact emotional representation learning. This paper develops an affective image analysis method based on the aesthetic fusion hybrid attention network (AFHA). A modular hybrid attention block is designed to extract image emotion features and model long-range dependencies of images. By stacking hybrid attention blocks in ResNet-style, we obtain an affective representation backbone. Furthermore, considering that image emotion is inseparable from aesthetics, we employ a modified ResNet to extract image aesthetics. Finally, through a fusion strategy, the image's emotion is considered with the aesthetics conveyed. Experiments demonstrate the close relationship between emotion and aesthetics, and our plan has an excellent competitive effect compared with existing methods on the image sentiment analysis dataset.

Citation Download Citation

Jixiong Ma, Hao Zhang, Kangjian He, and Dan Xu "Exploring affective image representation with visual attention and aesthetic fusion", Proc. SPIE 12705, Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022), 127051U (27 June 2023); https://doi.org/10.1117/12.2680098

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
10 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Emotion

Image analysis

Visualization

Convolution

Visual process modeling

Feature extraction

Autoregressive models

Show All Keywords

Keywords/Phrases

Search In:

Publication Years