20 March 2024 | Panchanand Jha · G. Praveen Kumar Yadav · Din Bandhu · Nuthalapati Hemalatha · Ravi Kumar Mandava · Mehmet Şükür Adin · Kuldeep K. Saxena · Mahaboo Patel
This research presents a real-time human–machine interaction framework for estimating and tracking human joints in a dynamic environment using a humanoid robot. The study employs a Kinect depth sensor and MediaPipe framework to obtain 3D joint position data of the human skeleton. The joint coordinates are then used to calculate joint angles via inverse kinematics. These angles are used to control the movement of a humanoid robot's neck, shoulder, and elbow using Python-Arduino serial communication. A comparison study was conducted between Kinect, MediaPipe, and real-time robot joint angles, revealing that the MediaPipe framework yields the minimum standard error compared to Kinect-based joint angles.
The research develops a real-time framework for obtaining various joint postures of a humanoid arm using a Kinect depth sensor and MediaPipe framework. It also implements an inverse kinematics approach to calculate joint angles for the humanoid arm. Standard error calculations are performed between joint angles obtained from inverse kinematics, Kinect depth sensor, and MediaPipe framework.
The study explores the application of human pose estimation (HPE) techniques in humanoid robotics, focusing on the use of deep learning models for 2D and 3D pose estimation. Various HPE frameworks, including OpenPose, HMR, OpenNI, VoxelNet, and PoseNet, are discussed. The MediaPipe framework is highlighted for its ability to produce minimal error and accurately classify joint landmarks. The authors also developed a 3D-printed robot prototype for implementing the HPE framework and tested it with both Kinect-based skeleton tracking and MediaPipe frameworks.
The study demonstrates the effectiveness of the MediaPipe framework in real-time human pose estimation and robotic control. The results show that the MediaPipe-based solution has a lower standard error compared to Kinect-based skeleton tracking. The research contributes to the field of human–robot interaction by providing a robust and accurate method for real-time joint angle estimation and control in humanoid robots. The findings suggest that the MediaPipe framework is a promising solution for real-time human–machine interaction in dynamic environments.This research presents a real-time human–machine interaction framework for estimating and tracking human joints in a dynamic environment using a humanoid robot. The study employs a Kinect depth sensor and MediaPipe framework to obtain 3D joint position data of the human skeleton. The joint coordinates are then used to calculate joint angles via inverse kinematics. These angles are used to control the movement of a humanoid robot's neck, shoulder, and elbow using Python-Arduino serial communication. A comparison study was conducted between Kinect, MediaPipe, and real-time robot joint angles, revealing that the MediaPipe framework yields the minimum standard error compared to Kinect-based joint angles.
The research develops a real-time framework for obtaining various joint postures of a humanoid arm using a Kinect depth sensor and MediaPipe framework. It also implements an inverse kinematics approach to calculate joint angles for the humanoid arm. Standard error calculations are performed between joint angles obtained from inverse kinematics, Kinect depth sensor, and MediaPipe framework.
The study explores the application of human pose estimation (HPE) techniques in humanoid robotics, focusing on the use of deep learning models for 2D and 3D pose estimation. Various HPE frameworks, including OpenPose, HMR, OpenNI, VoxelNet, and PoseNet, are discussed. The MediaPipe framework is highlighted for its ability to produce minimal error and accurately classify joint landmarks. The authors also developed a 3D-printed robot prototype for implementing the HPE framework and tested it with both Kinect-based skeleton tracking and MediaPipe frameworks.
The study demonstrates the effectiveness of the MediaPipe framework in real-time human pose estimation and robotic control. The results show that the MediaPipe-based solution has a lower standard error compared to Kinect-based skeleton tracking. The research contributes to the field of human–robot interaction by providing a robust and accurate method for real-time joint angle estimation and control in humanoid robots. The findings suggest that the MediaPipe framework is a promising solution for real-time human–machine interaction in dynamic environments.