NPGA: Neural Parametric Gaussian Avatars

NPGA: Neural Parametric Gaussian Avatars

29 May 2024 | SIMON GIEBENHAIN, Technical University of Munich, Germany; TOBIAS KIRSCHSTEIN, Technical University of Munich, Germany; MARTIN RÜNZ, Synthesia, Germany; LOURDES AGAPITO, University College London, United Kingdom; MATTHIAS NIESSNER, Technical University of Munich, Germany
**Neural Parametric Gaussian Avatars (NPGA)** is a novel method for creating high-fidelity, controllable avatars from multi-view video recordings. The approach leverages 3D Gaussian Splatting (3DGS) for efficient rendering and the rich expression space of Neural Parametric Head Models (NPHM) for fine-grained expression control. NPGA consists of a canonical Gaussian point cloud and a dynamics module that deforms the cloud using expression codes. Key contributions include a distillation strategy to utilize NPHM's expression prior, per-Gaussian latent features to enhance dynamic expressivity, and Laplacian smoothness terms to regularize the model. Evaluations on the NeFsemble dataset show that NPGA outperforms previous state-of-the-art methods by 2.6 PSNR and 0.021 SSIM in self-reenactment tasks, demonstrating accurate animation from real-world monocular videos. The method addresses limitations in controllability and reconstruction quality by extending the underlying 3DMM to include more body regions and leveraging large-scale multi-view datasets for improved learning.**Neural Parametric Gaussian Avatars (NPGA)** is a novel method for creating high-fidelity, controllable avatars from multi-view video recordings. The approach leverages 3D Gaussian Splatting (3DGS) for efficient rendering and the rich expression space of Neural Parametric Head Models (NPHM) for fine-grained expression control. NPGA consists of a canonical Gaussian point cloud and a dynamics module that deforms the cloud using expression codes. Key contributions include a distillation strategy to utilize NPHM's expression prior, per-Gaussian latent features to enhance dynamic expressivity, and Laplacian smoothness terms to regularize the model. Evaluations on the NeFsemble dataset show that NPGA outperforms previous state-of-the-art methods by 2.6 PSNR and 0.021 SSIM in self-reenactment tasks, demonstrating accurate animation from real-world monocular videos. The method addresses limitations in controllability and reconstruction quality by extending the underlying 3DMM to include more body regions and leveraging large-scale multi-view datasets for improved learning.
Reach us at info@study.space
[slides and audio] NPGA%3A Neural Parametric Gaussian Avatars