A new deepfake detection method by vision transformers

Ming Liao; Mingmei Chen

doi:10.1117/12.3051840

18 November 2024 A new deepfake detection method by vision transformers

Ming Liao, Mingmei Chen

Proceedings Volume 13403, International Conference on Algorithms, High Performance Computing, and Artificial Intelligence (AHPCAI 2024) ; 134033I (2024) https://doi.org/10.1117/12.3051840
Event: International Conference on Algorithms, High Performance Computing, and Artificial Intelligence, 2024, Zhengzhou, China

Abstract

Deepfake detection aims to authenticate the authenticity of facial images and videos, thereby providing crucial theoretical and technical support for portrait rights protection, false information identification, and online fraud prevention. Recent research has introduced a deep neural network Transformer, leveraging a self-attention mechanism to model long-range dependencies and global receptive fields. This advanced architecture enhances the ability to capture contextual relationships in images and temporal dependencies in videos, thereby improving the representational capacity of the detection system. In this paper, we propose a dual-branch feature extraction framework, incorporating a spatial domain feature extraction branch based on EfficientNet and a frequency domain feature extraction branch, to enrich the feature representation compared to single-branch methods. The extracted features are subsequently integrated with the Transformer's encoder structure and cross-attention mechanism, enabling effective modeling of feature correlations across global regions. Our proposed method achieves detection accuracies of 83.5%, 70.25%, and 78.5% for Deepfake, Face2Face, and NeuralTextures forged images, respectively.

(2024) Published by SPIE. Downloading of the abstract is permitted for personal use only.

Citation Download Citation

Ming Liao and Mingmei Chen "A new deepfake detection method by vision transformers", Proc. SPIE 13403, International Conference on Algorithms, High Performance Computing, and Artificial Intelligence (AHPCAI 2024) , 134033I (18 November 2024); https://doi.org/10.1117/12.3051840

ACCESS THE FULL ARTICLE

INSTITUTIONAL
Select your institution to access the SPIE Digital Library.

SELECT YOUR INSTITUTION

PERSONAL
Sign in with your SPIE account to access your personal subscriptions or to use specific features such as save to my library, sign up for alerts, save searches, etc.

PERSONAL SIGN IN

No SPIE Account? Create one

PURCHASE THIS CONTENT

SUBSCRIBE TO DIGITAL LIBRARY

50 downloads per 1-year subscription

Members: $195

Non-members: $335 ADD TO CART

25 downloads per 1 - year subscription

Members: $145

Non-members: $250 ADD TO CART

PURCHASE SINGLE ARTICLE

Includes PDF, HTML & Video, when available

Members: $17.00

Non-members: $21.00 ADD TO CART

PROCEEDINGS
5 PAGES

DOWNLOAD PAPER SAVE TO MY LIBRARY

GET CITATION

RIGHTS & PERMISSIONS

Get copyright permission Get copyright permission on Copyright Marketplace

KEYWORDS

Transformers

Feature extraction

Counterfeit detection

Data modeling

Object detection

Video

Machine learning

Show All Keywords

Keywords/Phrases

Search In:

Publication Years