Dynamic gesture recognition based on temporal shift module

Zhiqi Liu; Hua Li

doi:10.1117/12.2671435

28 April 2023 Dynamic gesture recognition based on temporal shift module

Zhiqi Liu, Hua Li

Proceedings Volume 12610, Third International Conference on Artificial Intelligence and Computer Engineering (ICAICE 2022); 126102O (2023) https://doi.org/10.1117/12.2671435
Event: Third International Conference on Artificial Intelligence and Computer Engineering (ICAICE 2022), 2022, Wuhan, China

Abstract

Dynamic gesture recognition is a very important interaction method in human-computer interaction. For the current research, multi-modal data and three-dimensional convolutional neural network are often used for training. Although the recognition accuracy is high and robustness is good, the amount of parameters is large and high computational cost. To solve the problem, a dynamic gesture recognition method based on Temporal Shift Module (TSM) is proposed on the basis of two-dimensional convolutional neural network. This method uses PyConvResNet-50 as the backbone network, adds the TSM module for information exchange in the time dimension, embeds the Motion Excitation module (ME) into the TSM to enhance short-term temporal modeling, and finally uses 2D-FCN for spatiotemporal feature fusion classification. The experimental results show that the recognition accuracy of the model on the large-scale gesture dataset Jester is 96.49%, which is comparable to that of the three-dimensional convolutional neural network, but the calculation amount is reduced by 63% as well. This method is suitable for the field of gesture recognition that requires high real-time performance.

Citation Download Citation

Zhiqi Liu and Hua Li "Dynamic gesture recognition based on temporal shift module", Proc. SPIE 12610, Third International Conference on Artificial Intelligence and Computer Engineering (ICAICE 2022), 126102O (28 April 2023); https://doi.org/10.1117/12.2671435

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available