Exploiting Style Latent Flows for Generalizing Deepfake Video Detection

Exploiting Style Latent Flows for Generalizing Deepfake Video Detection

2024 | Jongwook Choi, Taehoon Kim, Yonghyun Jeong, Seungryul Baek, Jongwon Choi
This paper proposes a novel deepfake video detection framework that leverages the temporal variations in style latent vectors to detect fake videos. The framework utilizes a StyleGRU module trained via contrastive learning to capture dynamic properties of style latent vectors, and a style attention module that integrates style-based features with content-based features to detect visual and temporal artifacts. The approach is evaluated across various benchmark scenarios, demonstrating superior performance in cross-dataset and cross-manipulation settings. The study shows that the temporal changes in style latent vectors are crucial for improving the generalization of deepfake detection. The framework includes a StyleGRU module that encodes temporal variations of style latent vectors into a style-based temporal feature, and a style attention module that combines this feature with content features to enhance detection accuracy. The Temporal Transformer Encoder (TTE) module maps the combined features into binary class labels (real/fake). The method is validated through extensive experiments, showing that the proposed approach outperforms existing methods in deepfake detection. The framework is effective in detecting deepfake videos across various datasets and manipulation types, and the results demonstrate the importance of style latent vector flow in generalizing deepfake detection. The study also highlights the potential of using style latent vectors for broader applications in deepfake detection beyond facial attributes.This paper proposes a novel deepfake video detection framework that leverages the temporal variations in style latent vectors to detect fake videos. The framework utilizes a StyleGRU module trained via contrastive learning to capture dynamic properties of style latent vectors, and a style attention module that integrates style-based features with content-based features to detect visual and temporal artifacts. The approach is evaluated across various benchmark scenarios, demonstrating superior performance in cross-dataset and cross-manipulation settings. The study shows that the temporal changes in style latent vectors are crucial for improving the generalization of deepfake detection. The framework includes a StyleGRU module that encodes temporal variations of style latent vectors into a style-based temporal feature, and a style attention module that combines this feature with content features to enhance detection accuracy. The Temporal Transformer Encoder (TTE) module maps the combined features into binary class labels (real/fake). The method is validated through extensive experiments, showing that the proposed approach outperforms existing methods in deepfake detection. The framework is effective in detecting deepfake videos across various datasets and manipulation types, and the results demonstrate the importance of style latent vector flow in generalizing deepfake detection. The study also highlights the potential of using style latent vectors for broader applications in deepfake detection beyond facial attributes.
Reach us at info@study.space
Understanding Exploiting Style Latent Flows for Generalizing Deepfake Video Detection