Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning

Bunny-VisionPro: Real-Time Bimanual Dexterous Teleoperation for Imitation Learning

3 Jul 2024 | Runyu Ding, Yuzhe Qin, Jiyue Zhu, Chengzhe Jia, Shiqi Yang, Ruihan Yang, Xiaojuan Qi, Xiaolong Wang
Bunny-VisionPro is a real-time bimanual dexterous teleoperation system designed to enhance human-robot interaction through immersive VR experiences. The system leverages the Apple Vision Pro to track operator hand movements and convert them into precise robotic commands, enabling intuitive control of high-DoF hand-arm systems. Key features include: 1. **Hand Motion Retargeting**: Maps operator finger poses to robot hand movements, handling loop joints efficiently. 2. **Arm Motion Control**:Computes joint trajectories for robot arms while considering collision avoidance and singularity management. 3. **Haptic Feedback**: Provides tactile sensations using low-cost Eccentric Rotating Mass (ERM) actuators, enhancing operator immersion and precision. The system outperforms existing teleoperation systems on the Telekinesis benchmark, achieving higher success rates and reduced task completion times. It also improves imitation learning performance, demonstrating better generalization to new scenarios. Notably, Bunny-VisionPro excels in complex, multi-stage, long-horizon dexterous manipulation tasks, making it a powerful tool for advancing dexterous manipulation and imitation learning.Bunny-VisionPro is a real-time bimanual dexterous teleoperation system designed to enhance human-robot interaction through immersive VR experiences. The system leverages the Apple Vision Pro to track operator hand movements and convert them into precise robotic commands, enabling intuitive control of high-DoF hand-arm systems. Key features include: 1. **Hand Motion Retargeting**: Maps operator finger poses to robot hand movements, handling loop joints efficiently. 2. **Arm Motion Control**:Computes joint trajectories for robot arms while considering collision avoidance and singularity management. 3. **Haptic Feedback**: Provides tactile sensations using low-cost Eccentric Rotating Mass (ERM) actuators, enhancing operator immersion and precision. The system outperforms existing teleoperation systems on the Telekinesis benchmark, achieving higher success rates and reduced task completion times. It also improves imitation learning performance, demonstrating better generalization to new scenarios. Notably, Bunny-VisionPro excels in complex, multi-stage, long-horizon dexterous manipulation tasks, making it a powerful tool for advancing dexterous manipulation and imitation learning.
Reach us at info@study.space
Understanding Bunny-VisionPro%3A Real-Time Bimanual Dexterous Teleoperation for Imitation Learning