Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting

Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting

2 Oct 2024 | Ziyi Yang, Xinyu Gao, Yang-Tian Sun, Yi-Hua Huang, Xiaoyang Lyu, Wen Zhou, Shaohui Jiao, Xiaojuan Qi, Xiaogang Jin
The paper introduces *Spec-Gaussian*, an advanced approach to 3D Gaussian Splattering (3D-GS) that enhances the rendering of specular and anisotropic components. The key contributions include: 1. **Anisotropic Spherical Gaussian (ASG) Appearance Field**: Instead of using spherical harmonics (SH), ASG is employed to model the view-dependent appearance of each 3D Gaussian, significantly improving the representation of high-frequency information such as specular highlights and anisotropy. 2. **Coarse-to-Fine Training Strategy**: This strategy helps eliminate floaters in real-world scenes by optimizing low-resolution rendering first, preventing the need to increase the number of 3D Gaussians and regularizing the learning process to avoid unnecessary geometric structures. 3. **Anchor-Based Gaussian Splatting**: To reduce storage overhead and acceleration, anchor-based Gaussian splatting is used, where anchor Gaussians guide the generation of neural Gaussians, reducing the computational burden. The method is evaluated on various datasets, including synthetic and real-world scenes, demonstrating superior performance in modeling complex specular and anisotropic features while maintaining fast rendering speed and balancing storage efficiency. The experimental results show that *Spec-Gaussian* outperforms existing methods in terms of rendering quality, as evidenced by quantitative metrics such as PSNR, SSIM, and LPIPS. The approach also effectively handles real-world scenes, removing floaters and improving the visual quality of specular highlights and anisotropy.The paper introduces *Spec-Gaussian*, an advanced approach to 3D Gaussian Splattering (3D-GS) that enhances the rendering of specular and anisotropic components. The key contributions include: 1. **Anisotropic Spherical Gaussian (ASG) Appearance Field**: Instead of using spherical harmonics (SH), ASG is employed to model the view-dependent appearance of each 3D Gaussian, significantly improving the representation of high-frequency information such as specular highlights and anisotropy. 2. **Coarse-to-Fine Training Strategy**: This strategy helps eliminate floaters in real-world scenes by optimizing low-resolution rendering first, preventing the need to increase the number of 3D Gaussians and regularizing the learning process to avoid unnecessary geometric structures. 3. **Anchor-Based Gaussian Splatting**: To reduce storage overhead and acceleration, anchor-based Gaussian splatting is used, where anchor Gaussians guide the generation of neural Gaussians, reducing the computational burden. The method is evaluated on various datasets, including synthetic and real-world scenes, demonstrating superior performance in modeling complex specular and anisotropic features while maintaining fast rendering speed and balancing storage efficiency. The experimental results show that *Spec-Gaussian* outperforms existing methods in terms of rendering quality, as evidenced by quantitative metrics such as PSNR, SSIM, and LPIPS. The approach also effectively handles real-world scenes, removing floaters and improving the visual quality of specular highlights and anisotropy.
Reach us at info@study.space
[slides and audio] Spec-Gaussian%3A Anisotropic View-Dependent Appearance for 3D Gaussian Splatting