Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer

Humanoid-Gym: Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer

18 May 2024 | Xinyang Gu, Yen-Jen Wang, Jianyu Chen
Humanoid-Gym is an open-source reinforcement learning (RL) framework based on Nvidia Isaac Gym, designed to train locomotion skills for humanoid robots with zero-shot transfer from simulation to the real world. It integrates a sim-to-sim framework to validate policies across different physical simulations, ensuring robustness and generalization. The framework has been verified on RobotEra's XBot-S and XBot-L humanoid robots in real-world environments with zero-shot sim-to-real transfer. Humanoid-Gym features specialized rewards and domain randomization techniques to simplify sim-to-real transfer. It also includes a sim-to-sim validation tool for testing policies across diverse environmental dynamics. The framework uses Proximal Policy Optimization (PPO) with asymmetric actor critic and privileged information during training. The reward function is structured into velocity tracking, gait reward, and regularization terms. The control policy operates at 100Hz with a high-frequency internal PD controller. The framework uses Isaac Gym for training and MuJoCo for sim-to-sim validation. The MuJoCo environment is calibrated to align with real-world dynamics. Experiments show that the adjusted MuJoCo simulation closely mirrors real-world dynamics, enabling researchers to validate policies through sim-to-sim, enhancing the potential for successful sim-to-real transfers. Humanoid-Gym provides a specialized reward function tailored for humanoid robotics, facilitating zero-shot transfer for robots of two distinct sizes.Humanoid-Gym is an open-source reinforcement learning (RL) framework based on Nvidia Isaac Gym, designed to train locomotion skills for humanoid robots with zero-shot transfer from simulation to the real world. It integrates a sim-to-sim framework to validate policies across different physical simulations, ensuring robustness and generalization. The framework has been verified on RobotEra's XBot-S and XBot-L humanoid robots in real-world environments with zero-shot sim-to-real transfer. Humanoid-Gym features specialized rewards and domain randomization techniques to simplify sim-to-real transfer. It also includes a sim-to-sim validation tool for testing policies across diverse environmental dynamics. The framework uses Proximal Policy Optimization (PPO) with asymmetric actor critic and privileged information during training. The reward function is structured into velocity tracking, gait reward, and regularization terms. The control policy operates at 100Hz with a high-frequency internal PD controller. The framework uses Isaac Gym for training and MuJoCo for sim-to-sim validation. The MuJoCo environment is calibrated to align with real-world dynamics. Experiments show that the adjusted MuJoCo simulation closely mirrors real-world dynamics, enabling researchers to validate policies through sim-to-sim, enhancing the potential for successful sim-to-real transfers. Humanoid-Gym provides a specialized reward function tailored for humanoid robotics, facilitating zero-shot transfer for robots of two distinct sizes.
Reach us at info@study.space
[slides] Humanoid-Gym%3A Reinforcement Learning for Humanoid Robot with Zero-Shot Sim2Real Transfer | StudySpace