Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learning

Diffusion Meets DAgger: Supercharging Eye-in-hand Imitation Learning

5 Jun 2024 | Xiaoyu Zhang, Matthew Chang, Pranav Kumar, Saurabh Gupta
Diffusion Meets DAgger (DMD) is a method that improves the sample efficiency of eye-in-hand imitation learning by using diffusion models to synthesize out-of-distribution states instead of collecting new data. This approach addresses the problem of compounding execution errors in imitation learning, where small errors lead to out-of-distribution states and poor performance. DMD uses a diffusion model to generate synthetic data, which is then used to train policies, leading to robust performance with few expert demonstrations. DMD was tested on four tasks: pushing, stacking, pouring, and hanging a shirt. In pushing, DMD achieved an 80% success rate with as few as 8 expert demonstrations, compared to 20% for behavior cloning (BC). In stacking, DMD succeeded on average 92% of the time across 5 cups, versus 40% for BC. When pouring coffee beans, DMD transferred to another cup successfully 80% of the time. Finally, DMD attained a 90% success rate for hanging a shirt on a clothing rack. DMD outperforms other methods, including a NeRF-based augmentation method, by 50% in the pushing task. It also shows improved performance in stacking and pouring tasks. The method uses a diffusion model to generate synthetic images and corresponding action labels, which are then used to train policies. The diffusion model is trained on task and play data, allowing it to generate realistic images even when the scene deforms during manipulation. The approach is effective in various settings, including 3DoF and 6DoF action spaces, generalization to new objects, and precision in reaching goal locations. DMD is also compatible with policies trained using diffusion models and improves performance in novel environments. The method has been tested on a variety of tasks and has shown significant improvements over traditional behavior cloning and other approaches. The results demonstrate the effectiveness of DMD in improving the sample efficiency and performance of eye-in-hand imitation learning.Diffusion Meets DAgger (DMD) is a method that improves the sample efficiency of eye-in-hand imitation learning by using diffusion models to synthesize out-of-distribution states instead of collecting new data. This approach addresses the problem of compounding execution errors in imitation learning, where small errors lead to out-of-distribution states and poor performance. DMD uses a diffusion model to generate synthetic data, which is then used to train policies, leading to robust performance with few expert demonstrations. DMD was tested on four tasks: pushing, stacking, pouring, and hanging a shirt. In pushing, DMD achieved an 80% success rate with as few as 8 expert demonstrations, compared to 20% for behavior cloning (BC). In stacking, DMD succeeded on average 92% of the time across 5 cups, versus 40% for BC. When pouring coffee beans, DMD transferred to another cup successfully 80% of the time. Finally, DMD attained a 90% success rate for hanging a shirt on a clothing rack. DMD outperforms other methods, including a NeRF-based augmentation method, by 50% in the pushing task. It also shows improved performance in stacking and pouring tasks. The method uses a diffusion model to generate synthetic images and corresponding action labels, which are then used to train policies. The diffusion model is trained on task and play data, allowing it to generate realistic images even when the scene deforms during manipulation. The approach is effective in various settings, including 3DoF and 6DoF action spaces, generalization to new objects, and precision in reaching goal locations. DMD is also compatible with policies trained using diffusion models and improves performance in novel environments. The method has been tested on a variety of tasks and has shown significant improvements over traditional behavior cloning and other approaches. The results demonstrate the effectiveness of DMD in improving the sample efficiency and performance of eye-in-hand imitation learning.
Reach us at info@study.space