DexDiffuser: Generating Dexterous Grasps with Diffusion Models

DexDiffuser: Generating Dexterous Grasps with Diffusion Models

6 Nov 2024 | Zehang Weng*, Haofei Lu*,†, Danica Kragic, and Jens Lundell
DexDiffuser is a novel dexterous grasping method that generates, evaluates, and refines grasps on partial object point clouds. It consists of two components: DexSampler, a conditional diffusion-based grasp sampler, and DexEvaluator, a dexterous grasp evaluator. DexSampler generates high-quality grasps by iteratively denoising randomly sampled grasps, conditioned on object point clouds. Two grasp refinement strategies, Evaluator-Guided Diffusion (EGD) and Evaluator-based Sampling Refinement (ESR), are introduced to improve grasp success probability. The method is trained on 1.7 million successful and unsuccessful Allegro Hand grasps generated using DexGraspNet across 5378 objects. Experimental results show that DexDiffuser outperforms the state-of-the-art multi-finger grasp generation method FFHNet with an average grasp success rate of 98.77% in simulation and 68.89% in real-world experiments, which is 11.63% and 20.00% higher, respectively. DexDiffuser is benchmarked against UniDexGrasp in simulation and FFHNet in both simulation and real-world settings. The method demonstrates superior performance in generating diverse and successful grasps, with DexSampler-BPS-ESR-2 achieving the best results in real-world experiments. DexDiffuser is evaluated on three visually and geometrically distinct object datasets in simulation and one dataset in the real world. The method is also compared with other approaches, including FFHNet and UniDexGrasp. DexDiffuser is effective in generating dexterous grasps on partial point clouds and is evaluated on real hardware, making it one of the few data-driven dexterous grasping methods that generate grasps directly on partial point clouds. The method's limitations include the sim-to-real gap, not accounting for environmental constraints, and relatively long grasp sampling time. Despite these limitations, DexDiffuser is a promising approach for dexterous manipulation tasks, including in-hand manipulation, dexterous grasping in clutter, and joint motion and dexterous grasp planning.DexDiffuser is a novel dexterous grasping method that generates, evaluates, and refines grasps on partial object point clouds. It consists of two components: DexSampler, a conditional diffusion-based grasp sampler, and DexEvaluator, a dexterous grasp evaluator. DexSampler generates high-quality grasps by iteratively denoising randomly sampled grasps, conditioned on object point clouds. Two grasp refinement strategies, Evaluator-Guided Diffusion (EGD) and Evaluator-based Sampling Refinement (ESR), are introduced to improve grasp success probability. The method is trained on 1.7 million successful and unsuccessful Allegro Hand grasps generated using DexGraspNet across 5378 objects. Experimental results show that DexDiffuser outperforms the state-of-the-art multi-finger grasp generation method FFHNet with an average grasp success rate of 98.77% in simulation and 68.89% in real-world experiments, which is 11.63% and 20.00% higher, respectively. DexDiffuser is benchmarked against UniDexGrasp in simulation and FFHNet in both simulation and real-world settings. The method demonstrates superior performance in generating diverse and successful grasps, with DexSampler-BPS-ESR-2 achieving the best results in real-world experiments. DexDiffuser is evaluated on three visually and geometrically distinct object datasets in simulation and one dataset in the real world. The method is also compared with other approaches, including FFHNet and UniDexGrasp. DexDiffuser is effective in generating dexterous grasps on partial point clouds and is evaluated on real hardware, making it one of the few data-driven dexterous grasping methods that generate grasps directly on partial point clouds. The method's limitations include the sim-to-real gap, not accounting for environmental constraints, and relatively long grasp sampling time. Despite these limitations, DexDiffuser is a promising approach for dexterous manipulation tasks, including in-hand manipulation, dexterous grasping in clutter, and joint motion and dexterous grasp planning.
Reach us at info@study.space