17 Jun 2024 | Shakiba Kheradmand, Daniel Rebain, Gopal Sharma, Weiwei Sun, Yang-Che Tseng, Hossam Isaac, Abhishek Kar, Andrea Tagliasacchi, Kwang Moo Yi
This paper addresses the limitations of 3D Gaussian Splatting (3DGS) in neural rendering, particularly the reliance on careful initialization and heuristic-based densification and pruning strategies. The authors propose a novel approach that reinterprets 3DGS as Markov Chain Monte Carlo (MCMC) sampling, where the set of 3D Gaussians is seen as random samples drawn from a probability distribution that describes the physical representation of the scene. By reformulating 3DGS updates as Stochastic Gradient Langevin Dynamics (SGLD), the method naturally explores the scene landscape and samples effective Gaussians. This approach eliminates the need for heuristics, such as cloning and splitting, and introduces a regularizer to encourage efficient use of Gaussians. The method is evaluated on various standard datasets, demonstrating improved rendering quality, robustness to initialization, and better control over the number of Gaussians. The contributions include a simpler optimization, principled relocation strategies, and enhanced robustness and quality in rendering.This paper addresses the limitations of 3D Gaussian Splatting (3DGS) in neural rendering, particularly the reliance on careful initialization and heuristic-based densification and pruning strategies. The authors propose a novel approach that reinterprets 3DGS as Markov Chain Monte Carlo (MCMC) sampling, where the set of 3D Gaussians is seen as random samples drawn from a probability distribution that describes the physical representation of the scene. By reformulating 3DGS updates as Stochastic Gradient Langevin Dynamics (SGLD), the method naturally explores the scene landscape and samples effective Gaussians. This approach eliminates the need for heuristics, such as cloning and splitting, and introduces a regularizer to encourage efficient use of Gaussians. The method is evaluated on various standard datasets, demonstrating improved rendering quality, robustness to initialization, and better control over the number of Gaussians. The contributions include a simpler optimization, principled relocation strategies, and enhanced robustness and quality in rendering.