Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots

Universal Manipulation Interface: In-The-Wild Robot Teaching Without In-The-Wild Robots

6 Mar 2024 | Cheng Chi*1,2, Zhenjia Xu*1,2, Chuer Pan1, Eric Cousineau3, Benjamin Burchfiel3, Siyuan Feng3, Russ Tedrake3, Shuran Song1,2
The Universal Manipulation Interface (UMI) is a portable, intuitive, and low-cost data collection and policy learning framework designed to transfer human demonstrations directly to deployable robot policies. UMI addresses the limitations of traditional teleoperation and unstructured human videos by using hand-held grippers with a carefully designed interface. The system captures diverse and rich data for challenging tasks such as dynamic, precise, bimanual, and long-horizon manipulation. UMI incorporates features like inference-time latency matching and relative-trajectory action representation to ensure hardware-agnostic policies. The learned policies are generalizable across multiple robot platforms and environments, achieving high success rates in out-of-distribution tests. UMI's hardware and software are open-sourced, making it accessible for researchers and enthusiasts to build and use. The paper discusses the design of the demonstration and policy interfaces, evaluates UMI's capabilities and generalization in real-world experiments, and highlights its advantages over traditional methods in terms of data collection efficiency and policy performance.The Universal Manipulation Interface (UMI) is a portable, intuitive, and low-cost data collection and policy learning framework designed to transfer human demonstrations directly to deployable robot policies. UMI addresses the limitations of traditional teleoperation and unstructured human videos by using hand-held grippers with a carefully designed interface. The system captures diverse and rich data for challenging tasks such as dynamic, precise, bimanual, and long-horizon manipulation. UMI incorporates features like inference-time latency matching and relative-trajectory action representation to ensure hardware-agnostic policies. The learned policies are generalizable across multiple robot platforms and environments, achieving high success rates in out-of-distribution tests. UMI's hardware and software are open-sourced, making it accessible for researchers and enthusiasts to build and use. The paper discusses the design of the demonstration and policy interfaces, evaluates UMI's capabilities and generalization in real-world experiments, and highlights its advantages over traditional methods in terms of data collection efficiency and policy performance.
Reach us at info@study.space