AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch

AnyRotate: Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch

3 Nov 2024 | Max Yang, Chenghua Lu, Alex Church, Yijiong Lin, Chris Ford, Haoran Li, Efi Psomopoulou, David A.W. Barton, Nathan F. Lepora
The paper introduces AnyRotate, a system for gravity-invariant multi-axis in-hand object rotation using dense featured sim-to-real touch. The authors address the challenge of achieving dexterous manipulation with multi-fingered robot hands by training a dense tactile policy in simulation and developing a method for rich tactile sensing to achieve zero-shot policy transfer to the real world. The system is designed to rotate unseen objects about arbitrary rotation axes in any hand direction, leveraging detailed contact information to improve performance. Key contributions include an RL formulation for multi-axis object rotation, a dense tactile representation, and an approach for sim-to-real tactile policy transfer. Experiments demonstrate the effectiveness of the system in handling objects with varying properties and show strong robustness across different hand orientations and rotation axes. The paper also discusses the importance of rich tactile sensing for dexterous manipulation and the limitations of the current approach, such as difficulties with objects having sharp geometric features.The paper introduces AnyRotate, a system for gravity-invariant multi-axis in-hand object rotation using dense featured sim-to-real touch. The authors address the challenge of achieving dexterous manipulation with multi-fingered robot hands by training a dense tactile policy in simulation and developing a method for rich tactile sensing to achieve zero-shot policy transfer to the real world. The system is designed to rotate unseen objects about arbitrary rotation axes in any hand direction, leveraging detailed contact information to improve performance. Key contributions include an RL formulation for multi-axis object rotation, a dense tactile representation, and an approach for sim-to-real tactile policy transfer. Experiments demonstrate the effectiveness of the system in handling objects with varying properties and show strong robustness across different hand orientations and rotation axes. The paper also discusses the importance of rich tactile sensing for dexterous manipulation and the limitations of the current approach, such as difficulties with objects having sharp geometric features.
Reach us at info@study.space
Understanding AnyRotate%3A Gravity-Invariant In-Hand Object Rotation with Sim-to-Real Touch