Language-driven Grasp Detection

Language-driven Grasp Detection

13 Jun 2024 | An Dinh Vuong, Minh Nhat Vu, Baoru Huang, Nghia Nguyen, Hieu Le, Thieu Vo, Anh Nguyen
The paper introduces Grasp-Anything++, a large-scale dataset for language-driven grasp detection, featuring 1 million samples, over 3 million objects, and 10 million grasping instructions. The dataset is created using foundation models to generate scene descriptions and grasp prompts, with a focus on providing detailed and descriptive information about the objects and their parts. The authors propose a diffusion-based method, LGD (Language-Driven Grasp Detection), which employs a contrastive training objective to improve the denoising process and enhance the performance of grasp detection. The method is evaluated using both vision-based metrics and real-world robotic experiments, demonstrating superior performance compared to state-of-the-art approaches. The paper also discusses the challenges and future directions for language-driven grasp detection, highlighting the potential of the Grasp-Anything++ dataset for various applications.The paper introduces Grasp-Anything++, a large-scale dataset for language-driven grasp detection, featuring 1 million samples, over 3 million objects, and 10 million grasping instructions. The dataset is created using foundation models to generate scene descriptions and grasp prompts, with a focus on providing detailed and descriptive information about the objects and their parts. The authors propose a diffusion-based method, LGD (Language-Driven Grasp Detection), which employs a contrastive training objective to improve the denoising process and enhance the performance of grasp detection. The method is evaluated using both vision-based metrics and real-world robotic experiments, demonstrating superior performance compared to state-of-the-art approaches. The paper also discusses the challenges and future directions for language-driven grasp detection, highlighting the potential of the Grasp-Anything++ dataset for various applications.
Reach us at info@study.space