Composable Part-Based Manipulation

Composable Part-Based Manipulation

9 May 2024 | Weiyu Liu, Jiayuan Mao, Joy Hsu, Tucker Hermans, Animesh Garg, Jiajun Wu
Composable Part-Based Manipulation (CPM) is a novel approach for robotic manipulation that leverages object-part decomposition and part-part correspondences to improve learning and generalization of manipulation skills. CPM uses a collection of composable diffusion models, where each model captures different inter-object correspondences. These models generate parameters for manipulation skills based on specific object parts, enabling strong generalization to novel objects and categories. The approach is validated in both simulated and real-world scenarios, demonstrating robust and generalized manipulation capabilities. CPM decomposes actions into part-based correspondences, allowing it to generalize across object categories and from simulation to the real world. The method uses part-based correspondences and task decomposition into distinct constraints to achieve this generalization. CPM is trained using diffusion models that learn primitive functional correspondences, which can be flexibly recombined during inference. The approach achieves strong generalization across various dimensions, including novel object instances and categories. The paper presents results on both PyBullet-based simulations and real-robot experiments, showing the effectiveness of CPM in achieving robust and generalized manipulation capabilities. CPM is compared to several baselines, including Transformer-BC, TAXPose, and PC-DDPM, and demonstrates superior performance in generalization tasks. The method is also tested in real-world scenarios, showing its transferability to real-world manipulation. The paper discusses limitations and future directions, including extending CPM to handle more objects and improving trajectory sampling techniques.Composable Part-Based Manipulation (CPM) is a novel approach for robotic manipulation that leverages object-part decomposition and part-part correspondences to improve learning and generalization of manipulation skills. CPM uses a collection of composable diffusion models, where each model captures different inter-object correspondences. These models generate parameters for manipulation skills based on specific object parts, enabling strong generalization to novel objects and categories. The approach is validated in both simulated and real-world scenarios, demonstrating robust and generalized manipulation capabilities. CPM decomposes actions into part-based correspondences, allowing it to generalize across object categories and from simulation to the real world. The method uses part-based correspondences and task decomposition into distinct constraints to achieve this generalization. CPM is trained using diffusion models that learn primitive functional correspondences, which can be flexibly recombined during inference. The approach achieves strong generalization across various dimensions, including novel object instances and categories. The paper presents results on both PyBullet-based simulations and real-robot experiments, showing the effectiveness of CPM in achieving robust and generalized manipulation capabilities. CPM is compared to several baselines, including Transformer-BC, TAXPose, and PC-DDPM, and demonstrates superior performance in generalization tasks. The method is also tested in real-world scenarios, showing its transferability to real-world manipulation. The paper discusses limitations and future directions, including extending CPM to handle more objects and improving trajectory sampling techniques.
Reach us at info@study.space
[slides and audio] Composable Part-Based Manipulation