December 3–6, 2024 | TOBIAS KIRSCHSTEIN, SIMON GIEBENHAIN, JIAPENG TANG, MARKOS GEORGOPOULOS, MATTHIAS NIESSNER
GGHead is a fast and generalizable 3D Gaussian Head generation method that can generate diverse 3D head representations from 2D images. The method uses 3D Gaussian Splatting within a 3D GAN framework to generate high-quality 3D heads in real-time at full resolution without the need for 2D super-resolution networks. The key idea is to use a template mesh's UV space to predict Gaussian attributes with a powerful 2D CNN, enabling efficient generation of 3D Gaussians. A novel UV total variation loss is introduced to improve geometric fidelity by enforcing continuity of rendered pixels with respect to their UV coordinates. This approach allows GGHead to generate 3D heads trained only from single-view 2D image observations, matching the quality of existing 3D head GANs on FFHQ while being significantly faster and fully 3D consistent. The method achieves real-time generation and rendering of high-quality 3D-consistent heads at 1024² resolution for the first time. GGHead outperforms other methods in terms of speed, 3D consistency, and geometric fidelity, and is highly scalable, enabling training at 1k resolutions while allowing real-time sample generation and head rendering. The method is evaluated on FFHQ and AFHQ datasets, showing strong performance in quantitative comparisons and qualitative results. It also demonstrates better surface reconstruction and 3D consistency compared to other methods. The method is efficient in terms of computational resources and memory usage, making it suitable for large-scale training and real-time applications. The method is robust to different template meshes and the number of Gaussians, and the UV total variation loss significantly improves the quality of generated 3D representations. The method has potential for future applications in 3D-consistent expression editing and training on large facial video datasets.GGHead is a fast and generalizable 3D Gaussian Head generation method that can generate diverse 3D head representations from 2D images. The method uses 3D Gaussian Splatting within a 3D GAN framework to generate high-quality 3D heads in real-time at full resolution without the need for 2D super-resolution networks. The key idea is to use a template mesh's UV space to predict Gaussian attributes with a powerful 2D CNN, enabling efficient generation of 3D Gaussians. A novel UV total variation loss is introduced to improve geometric fidelity by enforcing continuity of rendered pixels with respect to their UV coordinates. This approach allows GGHead to generate 3D heads trained only from single-view 2D image observations, matching the quality of existing 3D head GANs on FFHQ while being significantly faster and fully 3D consistent. The method achieves real-time generation and rendering of high-quality 3D-consistent heads at 1024² resolution for the first time. GGHead outperforms other methods in terms of speed, 3D consistency, and geometric fidelity, and is highly scalable, enabling training at 1k resolutions while allowing real-time sample generation and head rendering. The method is evaluated on FFHQ and AFHQ datasets, showing strong performance in quantitative comparisons and qualitative results. It also demonstrates better surface reconstruction and 3D consistency compared to other methods. The method is efficient in terms of computational resources and memory usage, making it suitable for large-scale training and real-time applications. The method is robust to different template meshes and the number of Gaussians, and the UV total variation loss significantly improves the quality of generated 3D representations. The method has potential for future applications in 3D-consistent expression editing and training on large facial video datasets.