Omnigrasp: Grasping Diverse Objects with Simulated Humanoids

Omnigrasp: Grasping Diverse Objects with Simulated Humanoids

18 May 2025 | Zhengyi Luo, Jinkun Cao, Sammy Christen, Alexander Winkler, Kris Kitani, Weipeng Xu
Omnigrasp is a method for controlling a simulated humanoid to grasp and carry objects along diverse trajectories. The method uses a pretrained universal dexterous motion representation to enable efficient learning of grasping policies. This representation allows the humanoid to pick up over 1200 objects and follow complex trajectories, with a focus on scalability and generalization. The key insight is to use a unified motion latent space that captures both body and hand movements, enabling the humanoid to perform precise and diverse object manipulations. The method does not require paired full-body motion data, and instead uses simple reward and state designs to train the policy. The policy is trained using randomly generated trajectories and only requires the object mesh and desired trajectories for grasping and transporting. The method achieves state-of-the-art success rates in following object trajectories and generalizing to unseen objects. The approach is evaluated on multiple datasets, including GRAB and OakInk, demonstrating its effectiveness in grasping a wide variety of objects and following complex trajectories. The method also shows robustness to input noise and has potential for real-world applications through sim-to-real modifications. The work contributes to the field of simulated humanoid control by introducing a scalable and generalizable approach for object manipulation.Omnigrasp is a method for controlling a simulated humanoid to grasp and carry objects along diverse trajectories. The method uses a pretrained universal dexterous motion representation to enable efficient learning of grasping policies. This representation allows the humanoid to pick up over 1200 objects and follow complex trajectories, with a focus on scalability and generalization. The key insight is to use a unified motion latent space that captures both body and hand movements, enabling the humanoid to perform precise and diverse object manipulations. The method does not require paired full-body motion data, and instead uses simple reward and state designs to train the policy. The policy is trained using randomly generated trajectories and only requires the object mesh and desired trajectories for grasping and transporting. The method achieves state-of-the-art success rates in following object trajectories and generalizing to unseen objects. The approach is evaluated on multiple datasets, including GRAB and OakInk, demonstrating its effectiveness in grasping a wide variety of objects and following complex trajectories. The method also shows robustness to input noise and has potential for real-world applications through sim-to-real modifications. The work contributes to the field of simulated humanoid control by introducing a scalable and generalizable approach for object manipulation.
Reach us at info@study.space