A lightweight real-time 3D hand gesture tracking solution for mobile devices

WenGang Han; Zhao Liu

doi:10.1117/12.3029630

13 May 2024 A lightweight real-time 3D hand gesture tracking solution for mobile devices

WenGang Han, Zhao Liu

Author Affiliations +

Proceedings Volume 13158, Seventh International Conference on Computer Graphics and Virtuality (ICCGV 2024); 1315809 (2024) https://doi.org/10.1117/12.3029630
Event: Seventh International Conference on Computer Graphics and Virtuality (ICCGV24), 2024, Hangzhou, China

Abstract

Gesture tracking is crucial for AR device human-computer interactions. Although many deep learning-based methods offer notable accuracy, their extensive parameters limit efficiency, challenging real-time deployments on low-power platforms. We present a lightweight, real-time 3D gesture tracking solution that determines hand positions and keypoints from a single RGB image in AR/VR devices. Using a two-stage algorithm, the initial stage identifies a hand's bounding frame. This frame then guides the second stage to detect 3D hand joint coordinates. These coordinates, once adjusted for camera parameters, yield the camera coordinates for hand keypoints. Our solution is optimized for low-power platforms, such as the RK3588 board, enabling real-time inferences with high detection quality (the speed performance of conventional models on RK3588 platform is illustrated in Table 1).

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

WenGang Han and Zhao Liu "A lightweight real-time 3D hand gesture tracking solution for mobile devices", Proc. SPIE 13158, Seventh International Conference on Computer Graphics and Virtuality (ICCGV 2024), 1315809 (13 May 2024); https://doi.org/10.1117/12.3029630

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

;

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
15 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

RGB color model

3D modeling

3D tracking

Convolution

Augmented reality

Mobile devices

Performance modeling

Show All Keywords

Keywords/Phrases

Search In:

Publication Years