Modality-inter fusion and enhancement network for dual-spectral object detection

Shuling Li; Jingxuan Jin; De Li

doi:10.1117/12.3030443

3 June 2024 Modality-inter fusion and enhancement network for dual-spectral object detection

Shuling Li, Jingxuan Jin, De Li

Proceedings Volume 13182, 2024 International Conference on Optoelectronic Information and Optical Engineering (OIOE 2024); 131820K (2024) https://doi.org/10.1117/12.3030443
Event: 2024 International Conference on Optoelectronic Information and Optical Engineering (OIOE 2024), 2024, Kunming, China

Abstract

In object detection tasks, when facing situations such as low contrast, occlusion, multiple overlapped targets, and changes in lighting, the detection performance of pure visible light (RGB) images or infrared (IR) images for object detection is not good. Combining image information from both visible light and infrared modalities can result in higher accuracy and more robust object detection under these challenges. The key to building an object detector based on the visible light and infrared modalities is how to merge the two modalities to obtain a more effective feature representation. For this purpose, we first constructed feature fusion module and feature enhancement module to effectively fusion and enhance the feature representations of the two modalities of visible light and infrared. Second, by combining the Bottleneck Attention Module (BAM) and the feature fusion and enhancement module, a dual-spectrum object detection network is constructed. Finally, we conducted experiments on the LLVIP and FLIR datasets, achieving mean Average Precision (mAP) of 46.2% on FLIR and 68.1% on LLVIP, surpassing the results of other recent methods. Experimental results showed that the dual-spectrum object detection network constructed in this paper effectively improved the performance of object detection.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Shuling Li, Jingxuan Jin, and De Li "Modality-inter fusion and enhancement network for dual-spectral object detection", Proc. SPIE 13182, 2024 International Conference on Optoelectronic Information and Optical Engineering (OIOE 2024), 131820K (3 June 2024); https://doi.org/10.1117/12.3030443

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
6 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Object detection

RGB color model

Feature fusion

Image fusion

Feature extraction

Infrared imaging

Infrared radiation

Show All Keywords

Keywords/Phrases

Search In:

Publication Years