Score-Guided Diffusion for 3D Human Recovery

Score-Guided Diffusion for 3D Human Recovery

14 Mar 2024 | Anastasis Stathopoulos, Ligong Han, Dimitris Metaxas
Score-Guided Human Mesh Recovery (ScoreHMR) is a novel approach for solving inverse problems in 3D human pose and shape reconstruction. Unlike traditional optimization methods, ScoreHMR uses a diffusion model to guide the denoising process, aligning the human body model with image observations through score guidance in the latent space of the diffusion model. The diffusion model is trained to capture the conditional distribution of human model parameters given an input image, enabling the method to solve various applications without retraining the diffusion model. ScoreHMR iteratively refines initial 3D estimates from regression networks using a diffusion model as a learned prior, achieving better alignment with observations. The method is evaluated on three settings: single-frame model fitting, multi-view reconstruction, and video-based human motion refinement. ScoreHMR consistently outperforms optimization baselines across all settings, demonstrating strong performance in challenging datasets. The approach is efficient, with fast runtime and effective alignment with image observations. ScoreHMR is available for further research and includes qualitative results on video sequences. The method leverages diffusion models to address inverse problems in 3D human recovery, offering a data-driven alternative to traditional optimization techniques.Score-Guided Human Mesh Recovery (ScoreHMR) is a novel approach for solving inverse problems in 3D human pose and shape reconstruction. Unlike traditional optimization methods, ScoreHMR uses a diffusion model to guide the denoising process, aligning the human body model with image observations through score guidance in the latent space of the diffusion model. The diffusion model is trained to capture the conditional distribution of human model parameters given an input image, enabling the method to solve various applications without retraining the diffusion model. ScoreHMR iteratively refines initial 3D estimates from regression networks using a diffusion model as a learned prior, achieving better alignment with observations. The method is evaluated on three settings: single-frame model fitting, multi-view reconstruction, and video-based human motion refinement. ScoreHMR consistently outperforms optimization baselines across all settings, demonstrating strong performance in challenging datasets. The approach is efficient, with fast runtime and effective alignment with image observations. ScoreHMR is available for further research and includes qualitative results on video sequences. The method leverages diffusion models to address inverse problems in 3D human recovery, offering a data-driven alternative to traditional optimization techniques.
Reach us at info@study.space
Understanding Score-Guided Diffusion for 3D Human Recovery