29 Feb 2024 | Jonathan Yang*, Catherine Glossop†, Arjun Bhorkar†, Dhruv Shah†, Quan Vuong‡, Chelsea Finn*, Dorsa Sadigh*, Sergey Levine†
This paper explores the limits of cross-embodiment learning in robotics, focusing on the transfer of knowledge between different types of robots and tasks. The authors train a single goal-conditioned policy to control a variety of robots, including robotic arms, drones, quadrupeds, and mobile bases, using data from both manipulation and navigation datasets. They find that co-training with navigation data enhances the policy's performance in manipulation tasks, achieving a 20% improvement over manipulation-only policies. Additionally, the policy demonstrates 5-7% improvement in navigation tasks when co-trained with manipulation data. The study also shows that navigation data helps manipulators understand their position relative to the goal, and that the policy can generalize to new robots, such as a mobile manipulator and a quadrotor, without additional training. The results highlight the potential of large-scale robotic policies to benefit from diverse datasets and cross-embodiment learning.This paper explores the limits of cross-embodiment learning in robotics, focusing on the transfer of knowledge between different types of robots and tasks. The authors train a single goal-conditioned policy to control a variety of robots, including robotic arms, drones, quadrupeds, and mobile bases, using data from both manipulation and navigation datasets. They find that co-training with navigation data enhances the policy's performance in manipulation tasks, achieving a 20% improvement over manipulation-only policies. Additionally, the policy demonstrates 5-7% improvement in navigation tasks when co-trained with manipulation data. The study also shows that navigation data helps manipulators understand their position relative to the goal, and that the policy can generalize to new robots, such as a mobile manipulator and a quadrotor, without additional training. The results highlight the potential of large-scale robotic policies to benefit from diverse datasets and cross-embodiment learning.