DexDiffuser: Generating Dexterous Grasps with Diffusion Models

DexDiffuser: Generating Dexterous Grasps with Diffusion Models

6 Nov 2024 | Zehang Weng*, Haofei Lu†, Danica Kragic, and Jens Lundell
DexDiffuser is a novel method for generating dexterous grasps using diffusion models. It includes a conditional diffusion-based grasp sampler, DexSampler, and a dexterous grasp evaluator, DexEvaluator. DexSampler generates high-quality grasps by iteratively denoising randomly sampled grasps conditioned on object point clouds. Two refinement strategies, Evaluator-Guided Diffusion (EGD) and Evaluator-based Sampling Refinement (ESR), are introduced to improve grasp success. Experimental results show that DexDiffuser outperforms FFHNet by 9.12% and 19.44% in simulation and real-world experiments, respectively. DexDiffuser is evaluated on three simulated datasets and one real-world dataset, achieving a grasp success rate of 98.77% in simulation and 68.89% in the real world. The method uses a Basis Point Set (BPS) encoding for point clouds and incorporates frequency encoding in the evaluator to better model grasp success. DexDiffuser is trained on 1.7 million grasps from 5378 objects, including both successful and unsuccessful grasps. The method is effective in generating diverse and successful grasps, with ESR and EGD improving grasp quality. DexDiffuser is evaluated on real robotic hardware, demonstrating its ability to handle noisy point clouds and real-world obstacles. The method's performance is lower in the real world due to the sim-to-real gap and environmental constraints. DexDiffuser is one of few data-driven methods that generate grasps directly on partial point clouds and is evaluated on real hardware. The method shows promise for solving other dexterous manipulation tasks, including in-hand manipulation and grasping in clutter.DexDiffuser is a novel method for generating dexterous grasps using diffusion models. It includes a conditional diffusion-based grasp sampler, DexSampler, and a dexterous grasp evaluator, DexEvaluator. DexSampler generates high-quality grasps by iteratively denoising randomly sampled grasps conditioned on object point clouds. Two refinement strategies, Evaluator-Guided Diffusion (EGD) and Evaluator-based Sampling Refinement (ESR), are introduced to improve grasp success. Experimental results show that DexDiffuser outperforms FFHNet by 9.12% and 19.44% in simulation and real-world experiments, respectively. DexDiffuser is evaluated on three simulated datasets and one real-world dataset, achieving a grasp success rate of 98.77% in simulation and 68.89% in the real world. The method uses a Basis Point Set (BPS) encoding for point clouds and incorporates frequency encoding in the evaluator to better model grasp success. DexDiffuser is trained on 1.7 million grasps from 5378 objects, including both successful and unsuccessful grasps. The method is effective in generating diverse and successful grasps, with ESR and EGD improving grasp quality. DexDiffuser is evaluated on real robotic hardware, demonstrating its ability to handle noisy point clouds and real-world obstacles. The method's performance is lower in the real world due to the sim-to-real gap and environmental constraints. DexDiffuser is one of few data-driven methods that generate grasps directly on partial point clouds and is evaluated on real hardware. The method shows promise for solving other dexterous manipulation tasks, including in-hand manipulation and grasping in clutter.
Reach us at info@study.space
[slides] DexDiffuser%3A Generating Dexterous Grasps With Diffusion Models | StudySpace