**OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation** **Authors:** Aadhithya Iyer, Zhuoran Peng, Yinlong Dai, Irmak Güzey, Siddhant Haldar, Soumith Chintala, Lerrel Pinto **Institution:** New York University, Meta **Abstract:** OPEN TEACH is an open-source, user-friendly teleoperation system designed to facilitate low-latency, high-frequency control of various robots, including multi-fingered hands and bimanual arms. Leveraging the Meta Quest 3 VR headset, users can perform intricate tasks with real-time visual feedback and smooth hand gesture control. The system supports both simulation and real-world environments, enabling the collection of high-quality demonstrations for policy learning. Experiments on 38 tasks across different robots show significant improvement over existing systems, with policies trained on collected data achieving an average success rate of 86% in both simulated and real-world settings. OPEN TEACH is fully open-source, promoting broader adoption and research in robotic manipulation. **Key Features:** - **Versatility:** Supports multiple arms, hands, and mobile manipulators. - **Calibration-Free:** Does not require specialized calibration. - **Low Latency:** Offers real-time control with 90Hz hand gesture detection. - **High-Frequency Visual Feedback:** Provides smooth visual feedback for intuitive control. - **User-Friendly:** Intuitive for both experienced and new users. **Experimental Evaluation:** - **Versatility:** Demonstrated across 38 tasks on different robots in both simulation and real-world environments. - **Policy Learning:** Policies trained on collected data achieve high success rates. - **Complex Tasks:** Capable of performing intricate, long-horizon tasks. - **User Study:** Showed significant improvement over existing systems in terms of success rates and completion times. **Limitations:** - Relies on the accuracy of hand pose detection in the VR headset. - May face challenges with finger occlusions and thumb retargeting. **Conclusion:** OPEN TEACH is a versatile and user-friendly teleoperation system that enhances the collection of high-quality demonstrations for robotic manipulation tasks. Its open-source nature and compatibility with various robots make it a valuable tool for researchers and practitioners in the field of robotics.**OPEN TEACH: A Versatile Teleoperation System for Robotic Manipulation** **Authors:** Aadhithya Iyer, Zhuoran Peng, Yinlong Dai, Irmak Güzey, Siddhant Haldar, Soumith Chintala, Lerrel Pinto **Institution:** New York University, Meta **Abstract:** OPEN TEACH is an open-source, user-friendly teleoperation system designed to facilitate low-latency, high-frequency control of various robots, including multi-fingered hands and bimanual arms. Leveraging the Meta Quest 3 VR headset, users can perform intricate tasks with real-time visual feedback and smooth hand gesture control. The system supports both simulation and real-world environments, enabling the collection of high-quality demonstrations for policy learning. Experiments on 38 tasks across different robots show significant improvement over existing systems, with policies trained on collected data achieving an average success rate of 86% in both simulated and real-world settings. OPEN TEACH is fully open-source, promoting broader adoption and research in robotic manipulation. **Key Features:** - **Versatility:** Supports multiple arms, hands, and mobile manipulators. - **Calibration-Free:** Does not require specialized calibration. - **Low Latency:** Offers real-time control with 90Hz hand gesture detection. - **High-Frequency Visual Feedback:** Provides smooth visual feedback for intuitive control. - **User-Friendly:** Intuitive for both experienced and new users. **Experimental Evaluation:** - **Versatility:** Demonstrated across 38 tasks on different robots in both simulation and real-world environments. - **Policy Learning:** Policies trained on collected data achieve high success rates. - **Complex Tasks:** Capable of performing intricate, long-horizon tasks. - **User Study:** Showed significant improvement over existing systems in terms of success rates and completion times. **Limitations:** - Relies on the accuracy of hand pose detection in the VR headset. - May face challenges with finger occlusions and thumb retargeting. **Conclusion:** OPEN TEACH is a versatile and user-friendly teleoperation system that enhances the collection of high-quality demonstrations for robotic manipulation tasks. Its open-source nature and compatibility with various robots make it a valuable tool for researchers and practitioners in the field of robotics.