TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation

TokenHMR: Advancing Human Mesh Recovery with a Tokenized Pose Representation

25 Apr 2024 | Sai Kumar Dwivedi, Yu Sun, Priyanka Patel, Yao Feng, Michael J. Black
TokenHMR is a novel approach to 3D human pose estimation from single images, addressing the fundamental trade-off between 3D and 2D accuracy. The method introduces a new loss function, *Threshold-Adaptive Loss Scaling* (TALS), which penalizes large 2D and pseudo-ground-truth (p-GT) errors while minimizing their impact on 3D pose estimation. Additionally, TokenHMR employs a token-based pose representation using a Vector Quantized-VAE (VQ-VAE) to discretize continuous poses, providing a prior over valid poses and improving robustness to occlusion. Extensive experiments on the EMDB and 3DPW datasets demonstrate that TokenHMR achieves state-of-the-art accuracy in 3D pose estimation, outperforming existing methods like HMR2.0 by 7.6% in Mean Per Joint Position Error (MPJPE). The method effectively leverages in-the-wild data while maintaining high 3D accuracy, making it a significant advancement in the field of 3D human pose estimation.TokenHMR is a novel approach to 3D human pose estimation from single images, addressing the fundamental trade-off between 3D and 2D accuracy. The method introduces a new loss function, *Threshold-Adaptive Loss Scaling* (TALS), which penalizes large 2D and pseudo-ground-truth (p-GT) errors while minimizing their impact on 3D pose estimation. Additionally, TokenHMR employs a token-based pose representation using a Vector Quantized-VAE (VQ-VAE) to discretize continuous poses, providing a prior over valid poses and improving robustness to occlusion. Extensive experiments on the EMDB and 3DPW datasets demonstrate that TokenHMR achieves state-of-the-art accuracy in 3D pose estimation, outperforming existing methods like HMR2.0 by 7.6% in Mean Per Joint Position Error (MPJPE). The method effectively leverages in-the-wild data while maintaining high 3D accuracy, making it a significant advancement in the field of 3D human pose estimation.
Reach us at info@study.space
[slides] TokenHMR%3A Advancing Human Mesh Recovery with a Tokenized Pose Representation | StudySpace