Hybrid Inverse Reinforcement Learning

Hybrid Inverse Reinforcement Learning

2024 | Juntao Ren, Gokul Swamy, Zhiwei Steven Wu, J. Andrew Bagnell, Sanjiban Choudhury
Hybrid inverse reinforcement learning (HIREL) addresses the high interaction complexity of standard inverse reinforcement learning (IRL) by combining online and expert data during policy search. This approach reduces unnecessary exploration by focusing the learner on states that align with expert behavior, leading to more sample-efficient learning. The key contribution is a reduction from IRL to expert-competitive RL, which allows for efficient policy search without requiring reset capabilities. Two hybrid IRL algorithms, Hype (model-free) and Hyper (model-based), are derived, both achieving strong performance guarantees. Empirically, Hype and Hyper outperform standard IRL and other baselines on continuous control tasks, demonstrating significant sample efficiency gains. The methods are applicable to a wide range of environments, including those with challenging exploration requirements. The work highlights the benefits of hybrid RL in imitation learning, offering a practical solution for real-world applications where reset capabilities are limited.Hybrid inverse reinforcement learning (HIREL) addresses the high interaction complexity of standard inverse reinforcement learning (IRL) by combining online and expert data during policy search. This approach reduces unnecessary exploration by focusing the learner on states that align with expert behavior, leading to more sample-efficient learning. The key contribution is a reduction from IRL to expert-competitive RL, which allows for efficient policy search without requiring reset capabilities. Two hybrid IRL algorithms, Hype (model-free) and Hyper (model-based), are derived, both achieving strong performance guarantees. Empirically, Hype and Hyper outperform standard IRL and other baselines on continuous control tasks, demonstrating significant sample efficiency gains. The methods are applicable to a wide range of environments, including those with challenging exploration requirements. The work highlights the benefits of hybrid RL in imitation learning, offering a practical solution for real-world applications where reset capabilities are limited.
Reach us at info@study.space
Understanding Hybrid Inverse Reinforcement Learning