Fusing SAR and EO imagery using CNN RGB-input channels, feature level, and decision level fusion

Noah S. Wood; Benjamin P. Lewis; Ram M. Narayanan

doi:10.1117/12.2622409

27 May 2022 Fusing SAR and EO imagery using CNN RGB-input channels, feature level, and decision level fusion

Noah S. Wood, Benjamin P. Lewis, Ram M. Narayanan

Proceedings Volume 12108, Radar Sensor Technology XXVI; 121080G (2022) https://doi.org/10.1117/12.2622409
Event: SPIE Defense + Commercial Sensing, 2022, Orlando, Florida, United States

Abstract

Within the field of target recognition, significant attention is given to data fusion techniques to optimize decision making in systems of multiple sensors. The challenge of fusing synthetic aperture radar (SAR) and electrooptical (EO) imagery is of particular interest to the defense community due to those sensors’ prevalence in target recognition systems. In this paper, the performances of two network architectures (a simple CNN and a ResNet) are compared, each implemented with multiple fusion methods to classify SAR and EO imagery of military targets. The Synthetic and Measured Paired Labeled Experiment (SAMPLE) dataset is used, an expansion of the MSTAR dataset, using both original measured SAR data and synthetic EO data. The classification performance of both networks is compared using the data modalities individually, using feature level fusion, using decision level fusion, and using a novel fusion method based on the three RGB-input channels of the ResNet (or other CNN for color image processing). In the input channel fusion method proposed, SAR imagery is fed to one of the three input channels, and the grayscale EO data is passed to a second of the three input channels. Despite its simplicity and off-the-shelf implementation, the input channel fusion method provides strong results, indicating it is worthy of further study.

Conference Presentation

Citation Download Citation

Noah S. Wood, Benjamin P. Lewis, and Ram M. Narayanan "Fusing SAR and EO imagery using CNN RGB-input channels, feature level, and decision level fusion", Proc. SPIE 12108, Radar Sensor Technology XXVI, 121080G (27 May 2022); https://doi.org/10.1117/12.2622409

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available