No-reference video quality assessment metric using spatiotemporal features through LSTM

Ngai-Wing Kwong; Sik-Ho Tsang; Yui-Lam Chan; Daniel Pak-Kong Lun; Tsz-Kwan Lee

doi:10.1117/12.2590406

13 March 2021 No-reference video quality assessment metric using spatiotemporal features through LSTM

Ngai-Wing Kwong, Sik-Ho Tsang, Yui-Lam Chan, Daniel Pak-Kong Lun, Tsz-Kwan Lee

Proceedings Volume 11766, International Workshop on Advanced Imaging Technology (IWAIT) 2021; 1176629 (2021) https://doi.org/10.1117/12.2590406
Event: International Workshop on Advanced Imaging Technology 2021 (IWAIT 2021), 2021, Online Only

Abstract

Nowadays, a precise video quality assessment (VQA) model is essential to maintain the quality of service (QoS). However, most existing VQA metrics are designed for specific purposes and ignore the spatiotemporal features of nature video. This paper proposes a novel general-purpose no-reference (NR) VQA metric adopting Long Short-Term Memory (LSTM) modules with the masking layer and pre-padding strategy, namely VQA-LSTM, to solve the above issues. First, we divide the distorted video into frames and extract some significant but also universal spatial and temporal features that could effectively reflect the quality of frames. Second, the data preprocessing stage and pre-padding strategy are used to process data to ease the training for our VQA-LSTM. Finally, a three-layer LSTM model incorporated with masking layer is designed to learn the sequence of spatial features as spatiotemporal features and learn the sequence of temporal features as the gradient of temporal features to evaluate the quality of videos. Two widely used VQA database, MCL-V and LIVE, are tested to prove the robustness of our VQA-LSTM, and the experimental results show that our VQA-LSTM has a better correlation with human perception than some state-of-the-art approaches.

Citation Download Citation

Ngai-Wing Kwong, Sik-Ho Tsang, Yui-Lam Chan, Daniel Pak-Kong Lun, and Tsz-Kwan Lee "No-reference video quality assessment metric using spatiotemporal features through LSTM", Proc. SPIE 11766, International Workshop on Advanced Imaging Technology (IWAIT) 2021, 1176629 (13 March 2021); https://doi.org/10.1117/12.2590406

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available