Human–machine interaction and implementation on the upper extremities of a humanoid robot

Human–machine interaction and implementation on the upper extremities of a humanoid robot

20 March 2024 | Panchanand Jha, G. Praveen Kumar Yadav, Din Bandhu, Nuthalapati Hemalatha, Ravi Kumar Mandava, Mehmet Şüküru Adin, Kuldeep K. Saxena, Mahaboob Patel
This paper presents a real-time human-machine interaction system for controlling the upper extremities of a humanoid robot using a human pose estimation framework. The authors utilize a Kinect depth sensor and the MediaPipe framework to obtain three-dimensional position information of human skeleton joints. The obtained joint coordinates are used to calculate joint angles through inverse kinematics, which are then communicated to the robot via Python-Arduino serial communication. The performance of the system is evaluated by comparing the joint angles obtained from the MediaPipe framework with those from the Kinect sensor and a real-time robot. The results show that the MediaPipe framework yields significantly lower standard errors compared to the Kinect-based method, demonstrating its effectiveness in real-time joint angle estimation. The study highlights the potential of MediaPipe in human-robot interaction applications, such as gesture-based control, medical rehabilitation, and assisting the elderly.This paper presents a real-time human-machine interaction system for controlling the upper extremities of a humanoid robot using a human pose estimation framework. The authors utilize a Kinect depth sensor and the MediaPipe framework to obtain three-dimensional position information of human skeleton joints. The obtained joint coordinates are used to calculate joint angles through inverse kinematics, which are then communicated to the robot via Python-Arduino serial communication. The performance of the system is evaluated by comparing the joint angles obtained from the MediaPipe framework with those from the Kinect sensor and a real-time robot. The results show that the MediaPipe framework yields significantly lower standard errors compared to the Kinect-based method, demonstrating its effectiveness in real-time joint angle estimation. The study highlights the potential of MediaPipe in human-robot interaction applications, such as gesture-based control, medical rehabilitation, and assisting the elderly.
Reach us at info@study.space
[slides and audio] Human%E2%80%93machine interaction and implementation on the upper extremities of a humanoid robot