Real-time object detection in 360-degree videos

Jounsup Park

doi:10.1117/12.2586403

12 April 2021 Real-time object detection in 360-degree videos

Jounsup Park

Proceedings Volume 11736, Real-Time Image Processing and Deep Learning 2021; 117360C (2021) https://doi.org/10.1117/12.2586403
Event: SPIE Defense + Commercial Sensing, 2021, Online Only

Abstract

Streaming of 360-degree videos over the internet is challenging task, but it provides rich multimedia experiences by allowing viewers to navigate 360-degree contents. The 360-degree videos need larger bandwidth and less latency to be streamed over the internet than the conventional videos. Therefore, non-visible area must be discarded from the video to save bandwidth. View prediction techniques have been used to predict visible area of the 360-degree video frames to be streamed. Linear regression using viewer’s past viewing behavior data is useful to predict short-term future behavior of the viewer, which is not useful when the network delay is longer than the prediction horizon. Object detection techniques help predicting viewers’ future motion for longer prediction horizon since the viewers tend to follow the objects that draw their attention. However, conventional object detection techniques using a convolutional neural network, such as YOLO, are difficult to be applied to 360-degree videos. There are distortions in the 360-degree videos when the spherical 360-degree video is projected into equi-rectangular videos for processing and storing purposes. A same object could have different shapes in the equi-rectangular video depends on their angular position in the sphere. Therefore, in this paper, we propose a multi-directional projection (MDP) technique to detect objects in the 360-degree videos. The proposed multi- directional projection technique mitigates the distortions in the equi-rectangular videos and feeds the redirected videos to the object detection system. Therefore, the neural network trained with conventional video dataset can be used without any change. Experimental result shows that the proposed method helps detecting objects in the edges of the 360-degree videos.

Conference Presentation

Citation Download Citation

Jounsup Park "Real-time object detection in 360-degree videos", Proc. SPIE 11736, Real-Time Image Processing and Deep Learning 2021, 117360C (12 April 2021); https://doi.org/10.1117/12.2586403

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available