22 Mar 2024 | Nick Heppert, Max Argus, Tim Welschehold, Thomas Brox, Abhinav Valada
The paper introduces DITTO (Demonstration Imitation by Trajectory Transformation), a novel method for teaching robots new tasks through one-shot imitation from a single human demonstration. The approach is divided into two stages: offline trajectory extraction and online trajectory generation. In the offline stage, the method segments and tracks objects in RGB-D videos to extract their trajectories. In the online stage, the method re-detects objects, warps the extracted trajectory to the current scene, and traces the trajectory with the robot. The method leverages various ancillary models, including segmentation, relative object pose estimation, and grasp prediction. The authors evaluate different combinations of correspondence and re-detection methods and validate their design decisions across a diverse range of tasks, including pick-and-place and articulated object manipulation. Extensive evaluations on a real robot system demonstrate the effectiveness and utility of the approach in real-world scenarios. The code and data are made publicly available at <http://ditto.cs.uni-freiburg.de>.The paper introduces DITTO (Demonstration Imitation by Trajectory Transformation), a novel method for teaching robots new tasks through one-shot imitation from a single human demonstration. The approach is divided into two stages: offline trajectory extraction and online trajectory generation. In the offline stage, the method segments and tracks objects in RGB-D videos to extract their trajectories. In the online stage, the method re-detects objects, warps the extracted trajectory to the current scene, and traces the trajectory with the robot. The method leverages various ancillary models, including segmentation, relative object pose estimation, and grasp prediction. The authors evaluate different combinations of correspondence and re-detection methods and validate their design decisions across a diverse range of tasks, including pick-and-place and articulated object manipulation. Extensive evaluations on a real robot system demonstrate the effectiveness and utility of the approach in real-world scenarios. The code and data are made publicly available at <http://ditto.cs.uni-freiburg.de>.