DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation

DexCap: Scalable and Portable Mocap Data Collection System for Dexterous Manipulation

2024-07-04 | Chen Wang, Haochen Shi, Weizhuo Wang, Ruohan Zhang, Li Fei-Fei, C. Karen Liu
DEXCAP is a portable and scalable hand motion capture system designed to facilitate the collection of high-quality human hand motion data for training dexterous robot manipulation skills. It leverages electromagnetic field (EMF) gloves and SLAM-based wrist pose tracking to provide precise, occlusion-resistant 3D hand motion data. DEXIL, a novel imitation learning algorithm, uses this data to train robot policies for complex manipulation tasks. The system includes an optional human-in-the-loop correction mechanism to refine policy performance. Extensive evaluations across six challenging tasks demonstrate the system's effectiveness in learning from in-the-wild data, showcasing its potential for advancing robotic dexterity. The hardware and software details, as well as experimental results, are provided to support the system's design and performance.DEXCAP is a portable and scalable hand motion capture system designed to facilitate the collection of high-quality human hand motion data for training dexterous robot manipulation skills. It leverages electromagnetic field (EMF) gloves and SLAM-based wrist pose tracking to provide precise, occlusion-resistant 3D hand motion data. DEXIL, a novel imitation learning algorithm, uses this data to train robot policies for complex manipulation tasks. The system includes an optional human-in-the-loop correction mechanism to refine policy performance. Extensive evaluations across six challenging tasks demonstrate the system's effectiveness in learning from in-the-wild data, showcasing its potential for advancing robotic dexterity. The hardware and software details, as well as experimental results, are provided to support the system's design and performance.
Reach us at info@study.space