25 Mar 2024 | Yun Liu, Haolin Yang, Xu Si, Ling Liu, Zipeng Li, Yuxiang Zhang, Yebin Liu, Li Yi
TACO is a large-scale bimanual hand-object manipulation dataset designed to support generalizable research in hand-object interaction. The dataset covers a wide range of tool-action-object combinations in real-world scenarios, including 2.5K motion sequences with precise 3D hand-object meshes, 2D segmentation, and realistic appearances. TACO is constructed using an automatic data acquisition pipeline that combines multi-view sensing and optical motion capture systems. The dataset supports three main tasks: compositional action recognition, generalizable hand-object motion forecasting, and cooperative grasp synthesis. Extensive experiments reveal new insights, challenges, and opportunities in generalizable hand-object interaction studies. The dataset and code are available at <https://taco2024.github.io>.TACO is a large-scale bimanual hand-object manipulation dataset designed to support generalizable research in hand-object interaction. The dataset covers a wide range of tool-action-object combinations in real-world scenarios, including 2.5K motion sequences with precise 3D hand-object meshes, 2D segmentation, and realistic appearances. TACO is constructed using an automatic data acquisition pipeline that combines multi-view sensing and optical motion capture systems. The dataset supports three main tasks: compositional action recognition, generalizable hand-object motion forecasting, and cooperative grasp synthesis. Extensive experiments reveal new insights, challenges, and opportunities in generalizable hand-object interaction studies. The dataset and code are available at <https://taco2024.github.io>.