In this paper, we present a method for temporal propagation of depth data that is available for so called key-frames
through video sequence. Our method requires that full frame depth information is assigned. Our method utilizes nearest
preceding and nearest following key-frames with known depth information. The propagation of depth information from
two sides is essential as it allows to solve most occlusion problems correctly. Image matching is based on the coherency
sensitive hashing (CSH) method and is done using image pyramids. Disclosed results are compared with temporal
interpolation based on motion vectors from optical flow algorithm. The proposed algorithm keeps sharp depth edges of
objects even in situations with fast motion or occlusions. It also handles well many situations, when the depth edges
don’t perfectly correspond with true edges of objects.
In this article we propose high quality motion estimation based on variational optical flow formulation with non-local
regularization term. To improve motion in occlusion areas we introduce occlusion motion inpainting based on 3-frame
motion clustering. Variational formulation of optical flow proved itself to be very successful, however a global
optimization of cost function can be time consuming. To achieve acceptable computation times we adapted the algorithm
that optimizes convex function in coarse-to-fine pyramid strategy and is suitable for modern GPU hardware
implementation. We also introduced two simplifications of cost function that significantly decrease computation time
with acceptable decrease of quality. For motion clustering based motion inpaitning in occlusion areas we introduce
effective method of occlusion aware joint 3-frame motion clustering using RANSAC algorithm. Occlusion areas are
inpainted by motion model taken from cluster that shows consistency in opposite direction. We tested our algorithm on
Middlebury optical flow benchmark, where we scored around 20th position, but being one of the fastest method near the
top. We also successfully used this algorithm in semi-automatic 2D to 3D conversion tool for spatio-temporal
background inpainting, automatic adaptive key frame detection and key points tracking.
Access to the requested content is limited to institutions that have purchased or subscribe to SPIE eBooks.
You are receiving this notice because your organization may not have SPIE eBooks access.*
*Shibboleth/Open Athens users─please
sign in
to access your institution's subscriptions.
To obtain this item, you may purchase the complete book in print or electronic format on
SPIE.org.
INSTITUTIONAL Select your institution to access the SPIE Digital Library.
PERSONAL Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.