The paper introduces Score-Guided Human Mesh Recovery (ScoreHMR), a novel approach for solving inverse problems in 3D human pose and shape reconstruction. ScoreHMR leverages diffusion models to align a human body model with image observations, particularly 2D keypoint detections. The method iteratively refines initial regression estimates using a diffusion model's denoising process guided by a task-specific score. This approach avoids the local minima issues of traditional optimization methods and captures complex data distributions effectively. ScoreHMR is evaluated on three settings: single-frame model fitting, multi-view refinement, and human motion refinement in video sequences. It consistently outperforms existing optimization baselines across various benchmarks, demonstrating its effectiveness and efficiency in refining 3D human reconstructions. The code and models are available on the project page.The paper introduces Score-Guided Human Mesh Recovery (ScoreHMR), a novel approach for solving inverse problems in 3D human pose and shape reconstruction. ScoreHMR leverages diffusion models to align a human body model with image observations, particularly 2D keypoint detections. The method iteratively refines initial regression estimates using a diffusion model's denoising process guided by a task-specific score. This approach avoids the local minima issues of traditional optimization methods and captures complex data distributions effectively. ScoreHMR is evaluated on three settings: single-frame model fitting, multi-view refinement, and human motion refinement in video sequences. It consistently outperforms existing optimization baselines across various benchmarks, demonstrating its effectiveness and efficiency in refining 3D human reconstructions. The code and models are available on the project page.