Real-time sign language recognition based on YOLO algorithm

Real-time sign language recognition based on YOLO algorithm

15 February 2024 | Melek Alaftekin · Ishak Pacal · Kenan Cicek
This study presents a real-time hand gesture recognition system for Turkish sign language based on the YOLOv4-CSP algorithm. The YOLOv4-CSP model is optimized by integrating CSPNet into the network architecture, enhancing learning ability and performance. The model uses the Mish activation function, CIoU loss function, and transformer block to improve detection speed and accuracy. The proposed model is trained on a dataset of Turkish sign language numbers and achieves high precision (98.95%), recall (98.15%), F1 score (98.55%), and mAP (99.49%) with a detection speed of 9.8 ms. The model outperforms other algorithms in real-time performance and accurate hand sign prediction, regardless of background. The study compares the proposed YOLOv4-CSP model with previous YOLO series models, including YOLOv3, YOLOv3-SPP, and YOLOv4-CSP. The results show that the proposed model has higher accuracy and speed than other models, making it suitable for real-time Turkish sign language recognition. The model is designed to detect static hand signals without the need for pre-image processing or a uniform background. The study also evaluates the performance of scratch training and transfer learning in object detection. The proposed YOLOv4-CSP model provides a more efficient and accurate solution for Turkish sign language recognition.This study presents a real-time hand gesture recognition system for Turkish sign language based on the YOLOv4-CSP algorithm. The YOLOv4-CSP model is optimized by integrating CSPNet into the network architecture, enhancing learning ability and performance. The model uses the Mish activation function, CIoU loss function, and transformer block to improve detection speed and accuracy. The proposed model is trained on a dataset of Turkish sign language numbers and achieves high precision (98.95%), recall (98.15%), F1 score (98.55%), and mAP (99.49%) with a detection speed of 9.8 ms. The model outperforms other algorithms in real-time performance and accurate hand sign prediction, regardless of background. The study compares the proposed YOLOv4-CSP model with previous YOLO series models, including YOLOv3, YOLOv3-SPP, and YOLOv4-CSP. The results show that the proposed model has higher accuracy and speed than other models, making it suitable for real-time Turkish sign language recognition. The model is designed to detect static hand signals without the need for pre-image processing or a uniform background. The study also evaluates the performance of scratch training and transfer learning in object detection. The proposed YOLOv4-CSP model provides a more efficient and accurate solution for Turkish sign language recognition.
Reach us at info@futurestudyspace.com