Multiple modality sensor fusion from synthetic aperture radar, lidar, and electro-optical systems using three-dimensional data representations

J. R. Jamora; Paul Sotirelis; Adam Nolan; Jeff Walrath; Rick Hubbard; Rajith Weerasinghe; Eric Young; Seth Young

doi:10.1117/12.2618297

31 May 2022 Multiple modality sensor fusion from synthetic aperture radar, lidar, and electro-optical systems using three-dimensional data representations

J. R. Jamora, Paul Sotirelis, Adam Nolan, Jeff Walrath, Rick Hubbard, Rajith Weerasinghe, Eric Young, Seth Young

Author Affiliations +

Proceedings Volume 12095, Algorithms for Synthetic Aperture Radar Imagery XXIX; 1209504 (2022) https://doi.org/10.1117/12.2618297
Event: SPIE Defense + Commercial Sensing, 2022, Orlando, Florida, United States

Abstract

Commonly, data exploitation for single sensors utilizes two-dimensional (2D) imagery. To best combine information from multiple sensing modalities, each with their own fundamental differences, we utilize sensor fusion to capture and leverage the inherent weaknesses from different sensing modalities. When fusing multiple sensor modalities together, this approach quickly becomes intractable as each sensor has unique projection planes and resolution. In this work, we present and analyze a data-driven approach for fusing multiple modalities by extracting data representations for each sensor into three-dimensional (3D) space, supporting sensor fusion natively in a common frame of reference. Photogrammetry and computer vision methods for recovering point clouds, such as structure from motion and multi-view stereo, from 2D electro-optical imagery has shown promising results. Additionally, 3D data representations can also be derived from interferometric synthetic aperture radar (IFSAR) and lidar sensors. We use point cloud representations for all three modalities, which allow us to leverage each sensing modality’s individual strengths and weaknesses. Given our data-driven focus, we emphasize fusing the point cloud data in controlled scenarios with known parameters. We also conduct an error analysis for each sensor modality based upon sensor position, resolution, and noise.

Conference Presentation

Citation Download Citation

J. R. Jamora, Paul Sotirelis, Adam Nolan, Jeff Walrath, Rick Hubbard, Rajith Weerasinghe, Eric Young, and Seth Young "Multiple modality sensor fusion from synthetic aperture radar, lidar, and electro-optical systems using three-dimensional data representations", Proc. SPIE 12095, Algorithms for Synthetic Aperture Radar Imagery XXIX, 1209504 (31 May 2022); https://doi.org/10.1117/12.2618297

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available