Multi-task learning using optical flow for motion magnification

Li Chen; Bingchao Zhao; Xiaoyue Liu; Zhaozhou Cai; Cong Peng

doi:10.1117/12.2680446

27 June 2023 Multi-task learning using optical flow for motion magnification

Li Chen, Bingchao Zhao, Xiaoyue Liu, Zhaozhou Cai, Cong Peng

Proceedings Volume 12705, Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022); 127052V (2023) https://doi.org/10.1117/12.2680446
Event: Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022), 2022, Nanjing, China

Abstract

The human visual systems cannot directly discern subtle but meaningful variations, such as the facial micro-expressions and the structural vibrations. Motion magnification techniques allow the naked eyes to perceive these variations. Current approaches on motion magnification generally belong to the Lagrangian and Eulerian perspectives. Nevertheless, the methods either require heavy computation or cannot extract tiny but valuable variations from noise disturbance. This paper utilizes multi-task learning to unite the Lagrangian and Eulerian perspectives for a novel motion magnification method. First, the method develops a multi-task network, which utilizes optical flow estimation to support motion magnification with accurate motion extraction. The homoscedastic uncertainty is then applied to balance task relation in the loss functions. In order to support multi-task learning, a simulated dataset is synthesized with real images from public datasets. Finally, the experimental results demonstrate that the proposed method exceeds the previous ones and optical flow can effectively support motion magnification.

Citation Download Citation

Li Chen, Bingchao Zhao, Xiaoyue Liu, Zhaozhou Cai, and Cong Peng "Multi-task learning using optical flow for motion magnification", Proc. SPIE 12705, Fourteenth International Conference on Graphics and Image Processing (ICGIP 2022), 127052V (27 June 2023); https://doi.org/10.1117/12.2680446

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available