Multiclass AI-Generated Deepfake Face Detection Using Patch-Wise Deep Learning Model

Multiclass AI-Generated Deepfake Face Detection Using Patch-Wise Deep Learning Model

21 January 2024 | Muhammad Asad Arshed, Shahzad Mumtaz, Muhammad Ibrahim, Christine Dewi, Muhammad Tanveer and Saeed Ahmed
This paper presents a novel approach for detecting multiclass AI-generated deepfakes using a patch-wise deep learning model based on Vision Transformers (ViTs). The study addresses the growing concern of deepfake content, which is generated using advanced AI techniques such as Generative Adversarial Networks (GANs) and Stable Diffusion. The primary objective is to evaluate the effectiveness of ViTs in detecting deepfake images compared to traditional Convolutional Neural Network (CNN)-based models. The research introduces a multiclass classification framework to address the challenges posed by deepfake generation methods like Stable Diffusion and StyleGAN2. The proposed method involves dividing the input image into patches and using ViT to extract global features, which enhances detection accuracy. The ViT model is trained on a multiclass dataset containing real images and deepfake images generated by Stable Diffusion and StyleGAN2. The dataset includes four classes: Real, GAN_Fake, Diffusion_Fake, and Stable&GAN_Fake, each with 10,000 samples. The model achieves a high detection accuracy of 99.90% on the multiclass-prepared dataset, outperforming state-of-the-art CNN-based models such as ResNet-50 and VGG-16. The study demonstrates that ViTs are effective in capturing global features and are robust to common manipulation techniques used in deepfake generation. The model's self-attention mechanism allows it to identify subtle inconsistencies and artifacts in deepfake images. The results highlight the potential of ViTs in combating deepfake threats and promoting trust and integrity in digital media. The research contributes to the field by introducing a multiclass deepfake detection approach, curating a specialized dataset, and showcasing the superior performance of the fine-tuned ViT model. The findings emphasize the importance of addressing the evolving landscape of deepfake creation and manipulation.This paper presents a novel approach for detecting multiclass AI-generated deepfakes using a patch-wise deep learning model based on Vision Transformers (ViTs). The study addresses the growing concern of deepfake content, which is generated using advanced AI techniques such as Generative Adversarial Networks (GANs) and Stable Diffusion. The primary objective is to evaluate the effectiveness of ViTs in detecting deepfake images compared to traditional Convolutional Neural Network (CNN)-based models. The research introduces a multiclass classification framework to address the challenges posed by deepfake generation methods like Stable Diffusion and StyleGAN2. The proposed method involves dividing the input image into patches and using ViT to extract global features, which enhances detection accuracy. The ViT model is trained on a multiclass dataset containing real images and deepfake images generated by Stable Diffusion and StyleGAN2. The dataset includes four classes: Real, GAN_Fake, Diffusion_Fake, and Stable&GAN_Fake, each with 10,000 samples. The model achieves a high detection accuracy of 99.90% on the multiclass-prepared dataset, outperforming state-of-the-art CNN-based models such as ResNet-50 and VGG-16. The study demonstrates that ViTs are effective in capturing global features and are robust to common manipulation techniques used in deepfake generation. The model's self-attention mechanism allows it to identify subtle inconsistencies and artifacts in deepfake images. The results highlight the potential of ViTs in combating deepfake threats and promoting trust and integrity in digital media. The research contributes to the field by introducing a multiclass deepfake detection approach, curating a specialized dataset, and showcasing the superior performance of the fine-tuned ViT model. The findings emphasize the importance of addressing the evolving landscape of deepfake creation and manipulation.
Reach us at info@study.space