Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects

Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects

6 Aug 2024 | Zicong Fan, Takehiko Ohkawa, Linlin Yang, Nie Lin, Zhishan Zhou, Shihao Zhou, Jiajun Liang, Zhong Gao, Xuanyang Zhang, Xue Zhang, Fei Li, Zheng Liu, Feng Lu, Karim Abou Zeid, Bastian Leibe, Jeongwan On, Seungryul Baek, Aditya Prakash, Saurabh Gupta, Kun He, Yoichi Sato, Otmar Hilliges, Hyung Jin Chang, Angela Yao
The paper introduces the HANDS23 challenge, which evaluates 3D hand and object reconstruction from egocentric viewpoints. The challenge is based on two datasets: AssemblyHands and ARCTIC. AssemblyHands focuses on 3D hand pose estimation from single-view images, while ARCTIC aims to reconstruct consistent motion of hands and articulated objects. The challenge highlights the difficulties of egocentric hand-object interactions, including heavy occlusions, camera distortion, and motion blur. The study analyzes the effectiveness of addressing these challenges through methods such as distortion correction, high-capacity transformers, and multi-view fusion. It also identifies remaining challenges, such as fast hand motion and object reconstruction from narrow views. The results show that recent methods significantly outperform baselines, with methods like JHands and PICO-AI achieving substantial improvements. The analysis reveals that egocentric views are more challenging than allocentric ones, and that multi-view fusion and model size play crucial roles in performance. The study contributes state-of-the-art baselines and insights for future research on egocentric hand-object interactions.The paper introduces the HANDS23 challenge, which evaluates 3D hand and object reconstruction from egocentric viewpoints. The challenge is based on two datasets: AssemblyHands and ARCTIC. AssemblyHands focuses on 3D hand pose estimation from single-view images, while ARCTIC aims to reconstruct consistent motion of hands and articulated objects. The challenge highlights the difficulties of egocentric hand-object interactions, including heavy occlusions, camera distortion, and motion blur. The study analyzes the effectiveness of addressing these challenges through methods such as distortion correction, high-capacity transformers, and multi-view fusion. It also identifies remaining challenges, such as fast hand motion and object reconstruction from narrow views. The results show that recent methods significantly outperform baselines, with methods like JHands and PICO-AI achieving substantial improvements. The analysis reveals that egocentric views are more challenging than allocentric ones, and that multi-view fusion and model size play crucial roles in performance. The study contributes state-of-the-art baselines and insights for future research on egocentric hand-object interactions.
Reach us at info@study.space
[slides] Benchmarks and Challenges in Pose Estimation for Egocentric Hand Interactions with Objects | StudySpace