18 Dec 2017 | Angjoo Kanazawa1,3, Michael J. Black2, David W. Jacobs3, Jitendra Malik1
The paper presents Human Mesh Recovery (HMR), an end-to-end framework for reconstructing a full 3D mesh of a human body from a single RGB image. Unlike most methods that focus on computing 2D or 3D joint locations, HMR produces a more comprehensive and useful mesh representation parameterized by shape and 3D joint angles. The main objective is to minimize the reprojection loss of keypoints, allowing the model to be trained using in-the-wild images with only ground truth 2D annotations. To address the under-constraint issue, the authors introduce an adversary trained to distinguish between real and fake 3D human bodies using a large database of 3D human meshes. This approach enables HMR to be trained with or without paired 2D-to-3D supervision, and it does not rely on intermediate 2D keypoint detection. The model runs in real-time given a bounding box containing the person and outperforms previous optimization-based methods in terms of 3D joint error and runtime. The paper also evaluates HMR on various tasks such as 3D joint location estimation and part segmentation, demonstrating competitive results.The paper presents Human Mesh Recovery (HMR), an end-to-end framework for reconstructing a full 3D mesh of a human body from a single RGB image. Unlike most methods that focus on computing 2D or 3D joint locations, HMR produces a more comprehensive and useful mesh representation parameterized by shape and 3D joint angles. The main objective is to minimize the reprojection loss of keypoints, allowing the model to be trained using in-the-wild images with only ground truth 2D annotations. To address the under-constraint issue, the authors introduce an adversary trained to distinguish between real and fake 3D human bodies using a large database of 3D human meshes. This approach enables HMR to be trained with or without paired 2D-to-3D supervision, and it does not rely on intermediate 2D keypoint detection. The model runs in real-time given a bounding box containing the person and outperforms previous optimization-based methods in terms of 3D joint error and runtime. The paper also evaluates HMR on various tasks such as 3D joint location estimation and part segmentation, demonstrating competitive results.