This paper proposes a deepfake detection system using a hybrid EfficientNet-Gated Recurrent Unit (GRU) network and EfficientNet-B0-based transfer learning, combined with a novel Particle Swarm Optimization (PSO) algorithm for hyperparameter selection. The system consists of three key steps: (1) data preprocessing for facial region extraction, (2) PSO-based hyperparameter optimization during network training, and (3) model establishment using the selected optimal settings for video classification. The proposed PSO algorithm incorporates composite leader generation and reinforcement learning-based search strategy allocation to mitigate premature convergence. The hybrid EfficientNet-GRU model and EfficientNet-B0 model are trained on datasets containing deepfake and genuine videos to classify fake and real videos. The empirical results show that the PSO-based EfficientNet-GRU and EfficientNet-B0 networks outperform existing methods for several deepfake datasets. The PSO algorithm is also effective in solving a variety of unimodal and multimodal benchmark functions. The research contributes to the field of deepfake detection by introducing a novel PSO-based hyperparameter search method and a hybrid CNN-RNN model for video forgery classification. The proposed system demonstrates superior performance in detecting deepfakes compared to existing methods.This paper proposes a deepfake detection system using a hybrid EfficientNet-Gated Recurrent Unit (GRU) network and EfficientNet-B0-based transfer learning, combined with a novel Particle Swarm Optimization (PSO) algorithm for hyperparameter selection. The system consists of three key steps: (1) data preprocessing for facial region extraction, (2) PSO-based hyperparameter optimization during network training, and (3) model establishment using the selected optimal settings for video classification. The proposed PSO algorithm incorporates composite leader generation and reinforcement learning-based search strategy allocation to mitigate premature convergence. The hybrid EfficientNet-GRU model and EfficientNet-B0 model are trained on datasets containing deepfake and genuine videos to classify fake and real videos. The empirical results show that the PSO-based EfficientNet-GRU and EfficientNet-B0 networks outperform existing methods for several deepfake datasets. The PSO algorithm is also effective in solving a variety of unimodal and multimodal benchmark functions. The research contributes to the field of deepfake detection by introducing a novel PSO-based hyperparameter search method and a hybrid CNN-RNN model for video forgery classification. The proposed system demonstrates superior performance in detecting deepfakes compared to existing methods.