Bunny-VisionPro is a real-time bimanual dexterous teleoperation system designed to enhance human-robot interaction through immersive VR experiences. The system leverages the Apple Vision Pro to track operator hand movements and convert them into precise robotic commands, enabling intuitive control of high-DoF hand-arm systems. Key features include:
1. **Hand Motion Retargeting**: Maps operator finger poses to robot hand movements, handling loop joints efficiently.
2. **Arm Motion Control**:Computes joint trajectories for robot arms while considering collision avoidance and singularity management.
3. **Haptic Feedback**: Provides tactile sensations using low-cost Eccentric Rotating Mass (ERM) actuators, enhancing operator immersion and precision.
The system outperforms existing teleoperation systems on the Telekinesis benchmark, achieving higher success rates and reduced task completion times. It also improves imitation learning performance, demonstrating better generalization to new scenarios. Notably, Bunny-VisionPro excels in complex, multi-stage, long-horizon dexterous manipulation tasks, making it a powerful tool for advancing dexterous manipulation and imitation learning.Bunny-VisionPro is a real-time bimanual dexterous teleoperation system designed to enhance human-robot interaction through immersive VR experiences. The system leverages the Apple Vision Pro to track operator hand movements and convert them into precise robotic commands, enabling intuitive control of high-DoF hand-arm systems. Key features include:
1. **Hand Motion Retargeting**: Maps operator finger poses to robot hand movements, handling loop joints efficiently.
2. **Arm Motion Control**:Computes joint trajectories for robot arms while considering collision avoidance and singularity management.
3. **Haptic Feedback**: Provides tactile sensations using low-cost Eccentric Rotating Mass (ERM) actuators, enhancing operator immersion and precision.
The system outperforms existing teleoperation systems on the Telekinesis benchmark, achieving higher success rates and reduced task completion times. It also improves imitation learning performance, demonstrating better generalization to new scenarios. Notably, Bunny-VisionPro excels in complex, multi-stage, long-horizon dexterous manipulation tasks, making it a powerful tool for advancing dexterous manipulation and imitation learning.