RialTo is a system designed to robustify real-world imitation learning policies using reinforcement learning in "digital twin" simulation environments. The system aims to address the challenges of learning robust behaviors without requiring extensive real-world data collection or human supervision. Key contributions include: 1. **Real-to-Sim-to-Real Pipeline**: RialTo constructs realistic simulation environments from real-world data, transfers real-world demonstrations to these simulations, and fine-tunes policies using reinforcement learning. This process is automated and requires minimal human effort in environment design and reward engineering. 2. **Inverse Distillation**: A novel technique to transfer real-world demonstrations into simulations, enabling efficient fine-tuning with sparse rewards and improved policy robustness. 3. **Graphical Interface**: An intuitive tool for quickly scanning and constructing digital twins of real-world scenes, facilitating the creation of accurate and realistic simulation environments. 4. **Robust Policy Learning**: RialTo learns policies that perform diverse manipulation tasks in the real world, showing robustness to disturbances and distractors. The system evaluates across various tasks, demonstrating a 67% improvement in average success rate over baselines. 5. **Experiments**: Extensive evaluations on eight tasks, including 6-DoF grasping and reorientation, show that RialTo outperforms imitation learning and other baselines, achieving high success rates even under challenging conditions. 6. **User Study**: A user study confirms the usability and efficiency of RialTo's pipeline for scene and policy transfer, indicating that the process is intuitive and does not require significant time or labor. RialTo addresses the limitations of traditional methods by combining the strengths of imitation learning and reinforcement learning, making it a scalable and robust solution for real-world robotic manipulation tasks.RialTo is a system designed to robustify real-world imitation learning policies using reinforcement learning in "digital twin" simulation environments. The system aims to address the challenges of learning robust behaviors without requiring extensive real-world data collection or human supervision. Key contributions include: 1. **Real-to-Sim-to-Real Pipeline**: RialTo constructs realistic simulation environments from real-world data, transfers real-world demonstrations to these simulations, and fine-tunes policies using reinforcement learning. This process is automated and requires minimal human effort in environment design and reward engineering. 2. **Inverse Distillation**: A novel technique to transfer real-world demonstrations into simulations, enabling efficient fine-tuning with sparse rewards and improved policy robustness. 3. **Graphical Interface**: An intuitive tool for quickly scanning and constructing digital twins of real-world scenes, facilitating the creation of accurate and realistic simulation environments. 4. **Robust Policy Learning**: RialTo learns policies that perform diverse manipulation tasks in the real world, showing robustness to disturbances and distractors. The system evaluates across various tasks, demonstrating a 67% improvement in average success rate over baselines. 5. **Experiments**: Extensive evaluations on eight tasks, including 6-DoF grasping and reorientation, show that RialTo outperforms imitation learning and other baselines, achieving high success rates even under challenging conditions. 6. **User Study**: A user study confirms the usability and efficiency of RialTo's pipeline for scene and policy transfer, indicating that the process is intuitive and does not require significant time or labor. RialTo addresses the limitations of traditional methods by combining the strengths of imitation learning and reinforcement learning, making it a scalable and robust solution for real-world robotic manipulation tasks.