27 Mar 2024 | Zhi Su*,2, Xiaoyu Huang*,1, Daniel Ordoñez-Apraza3, Yunfei Li2, Zhongyu Li1, Qiayuan Liao1 Giulio Turrisi3, Massimiliano Pontil3, Claudio Semini3, Yi Wu2,4, Koushil Sreenath1
This paper explores the benefits of leveraging symmetry in model-free reinforcement learning (RL) for legged locomotion control. The authors investigate two approaches to incorporate symmetry: modifying network architectures to be strictly equivariant/invariant and using data augmentation to approximate equivariant/invariant actor-critics. The methods are applied to challenging loco-manipulation and bipedal locomotion tasks on quadrupedal robots, and compared with an unconstrained baseline. The results show that strictly equivariant policies consistently outperform other methods in terms of sample efficiency and task performance in simulation. Additionally, symmetry-incorporated approaches exhibit better gait quality, higher robustness, and can be deployed zero-shot in real-world experiments. The paper also discusses the importance of symmetry constraints in improving the performance of RL algorithms, especially in tasks with morphological symmetries.This paper explores the benefits of leveraging symmetry in model-free reinforcement learning (RL) for legged locomotion control. The authors investigate two approaches to incorporate symmetry: modifying network architectures to be strictly equivariant/invariant and using data augmentation to approximate equivariant/invariant actor-critics. The methods are applied to challenging loco-manipulation and bipedal locomotion tasks on quadrupedal robots, and compared with an unconstrained baseline. The results show that strictly equivariant policies consistently outperform other methods in terms of sample efficiency and task performance in simulation. Additionally, symmetry-incorporated approaches exhibit better gait quality, higher robustness, and can be deployed zero-shot in real-world experiments. The paper also discusses the importance of symmetry constraints in improving the performance of RL algorithms, especially in tasks with morphological symmetries.