2 Nov 2024 | Minghuan Liu*, Zixuan Chen*, Xuxin Cheng, Yandong Ji Rizhao Qiu, Ruihan Yang, Xiaolong Wang
The paper presents a framework for legged loco-manipulation, named Visual Whole-Body Control (VBC), which enables a quadruped robot equipped with an arm to perform manipulation tasks autonomously. VBC consists of a low-level control policy and a high-level task-planning policy. The low-level policy tracks body velocities and end-effector positions using all degrees of freedom, while the high-level policy proposes velocities and end-effector positions based on visual inputs. The framework is trained in simulation and deployed on real robots without real-world data collection or fine-tuning. Extensive experiments demonstrate the effectiveness of VBC in picking up diverse objects at varying heights and locations, showing superior performance over baselines. The system's autonomous behavior and ability to adapt to different environments are highlighted, along with its robustness and generalization capabilities.The paper presents a framework for legged loco-manipulation, named Visual Whole-Body Control (VBC), which enables a quadruped robot equipped with an arm to perform manipulation tasks autonomously. VBC consists of a low-level control policy and a high-level task-planning policy. The low-level policy tracks body velocities and end-effector positions using all degrees of freedom, while the high-level policy proposes velocities and end-effector positions based on visual inputs. The framework is trained in simulation and deployed on real robots without real-world data collection or fine-tuning. Extensive experiments demonstrate the effectiveness of VBC in picking up diverse objects at varying heights and locations, showing superior performance over baselines. The system's autonomous behavior and ability to adapt to different environments are highlighted, along with its robustness and generalization capabilities.