2 May 2024 | Ryan Hoque, Ajay Mandlekar, Caelan Garrett, Ken Goldberg, Dieter Fox
IntervenGen (I-Gen) is a novel data generation system designed to address distribution shift in imitation learning (IL) for robot control policies. Distribution shift occurs when the conditions at evaluation time differ from those in the training data, leading to poor policy performance. I-Gen aims to increase policy robustness by autonomously generating a large set of corrective interventions from a small number of human interventions. The system leverages closed-loop policy execution and open-loop trajectory replay to broaden the distribution of visited mistake states, allowing for the generation of diverse and rich coverage of potential policy mistakes.
The authors apply I-Gen to four simulated environments and one physical environment, focusing on improving policy robustness against object pose estimation errors. I-Gen demonstrates significant improvements in policy robustness, achieving up to 39× better performance with only 10 human interventions. The system outperforms other methods, including human demonstrations and MimicGen, which is a similar data generation system. I-Gen also facilitates sim-to-real transfer, showing that policies trained in simulation retain robustness to erroneous state estimation when deployed in the real world.
The contributions of I-Gen include its ability to generate interventional data, its application to improve policy robustness in high-precision manipulation tasks, and its utility across different environments and sources of observation error. The system's effectiveness is demonstrated through experiments and comparisons with various baselines, highlighting its potential for reducing the burden on human operators and enhancing the robustness of robot policies.IntervenGen (I-Gen) is a novel data generation system designed to address distribution shift in imitation learning (IL) for robot control policies. Distribution shift occurs when the conditions at evaluation time differ from those in the training data, leading to poor policy performance. I-Gen aims to increase policy robustness by autonomously generating a large set of corrective interventions from a small number of human interventions. The system leverages closed-loop policy execution and open-loop trajectory replay to broaden the distribution of visited mistake states, allowing for the generation of diverse and rich coverage of potential policy mistakes.
The authors apply I-Gen to four simulated environments and one physical environment, focusing on improving policy robustness against object pose estimation errors. I-Gen demonstrates significant improvements in policy robustness, achieving up to 39× better performance with only 10 human interventions. The system outperforms other methods, including human demonstrations and MimicGen, which is a similar data generation system. I-Gen also facilitates sim-to-real transfer, showing that policies trained in simulation retain robustness to erroneous state estimation when deployed in the real world.
The contributions of I-Gen include its ability to generate interventional data, its application to improve policy robustness in high-precision manipulation tasks, and its utility across different environments and sources of observation error. The system's effectiveness is demonstrated through experiments and comparisons with various baselines, highlighting its potential for reducing the burden on human operators and enhancing the robustness of robot policies.