20 December 2019 Photorealistic image reconstruction from hybrid intensity and event-based sensor
Prasan Shedligeri, Kaushik Mitra
Author Affiliations +
Abstract

Event sensors output a stream of asynchronous brightness changes (called “events”) at a very high temporal rate. Previous works on recovering the lost intensity information from the event sensor data have heavily relied on the event stream, which makes the reconstructed images nonphotorealistic and also susceptible to noise in the event stream. A method is proposed to reconstruct photorealistic intensity images from a hybrid sensor consisting of a low-frame-rate conventional camera and the event sensor. The texture-rich information from a traditional image sensor with the motion-rich information from the event sensor is exploited, producing photorealistic high-frame-rate videos. To accomplish the task, the low-frame-rate intensity images are warped to temporally dense locations of the event data. The results obtained from the proposed algorithm are more photorealistic compared to any of the previous state-of-the-art algorithms. The algorithm’s robustness to abrupt camera motion and noise in the event sensor data is also demonstrated.

© 2019 SPIE and IS&T 1017-9909/2019/$28.00 © 2019 SPIE and IS&T
Prasan Shedligeri and Kaushik Mitra "Photorealistic image reconstruction from hybrid intensity and event-based sensor," Journal of Electronic Imaging 28(6), 063012 (20 December 2019). https://doi.org/10.1117/1.JEI.28.6.063012
Received: 24 May 2019; Accepted: 4 December 2019; Published: 20 December 2019
Lens.org Logo
CITATIONS
Cited by 31 scholarly publications.
Advertisement
Advertisement
RIGHTS & PERMISSIONS
Get copyright permission  Get copyright permission on Copyright Marketplace
KEYWORDS
Sensors

Image sensors

Image restoration

Reconstruction algorithms

Cameras

Video

Optical flow

RELATED CONTENT

Dynamic turbulence mitigation with large moving objects
Proceedings of SPIE (October 06 2017)
On-the-move independently moving target detection
Proceedings of SPIE (May 02 2007)
Motion blur removal in nonlinear sensors
Proceedings of SPIE (January 27 2010)
Gain fixed-pattern-noise correction via optical flow
Proceedings of SPIE (April 24 2002)

Back to Top