StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering

StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering

July 2024 | LUKAS RADL, MICHAEL STEINER, MATHIAS PARGER, ALEXANDER WEINRAUCH, BERNHARD KERBL, MARKUS STEINBERGER
StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering This paper presents a novel hierarchical rasterization approach for 3D Gaussian Splatting (3DGS) to address popping artifacts and view inconsistencies during camera rotations. The main challenge in 3DGS is the approximate global sorting scheme that leads to popping artifacts during view rotation. The proposed method introduces a hierarchical per-pixel sorting strategy to effectively circumvent both short-range popping artifacts and long-range view inconsistencies. The paper discusses the limitations of 3DGS, which relies on a global sort based on the view-space z-coordinate of each Gaussian's mean. This approach leads to popping artifacts during camera rotations due to changes in the relative depth of shells. The proposed method improves rendering consistency by computing the optimal depth for each Gaussian along the view ray, which allows for accurate sorting and eliminates popping artifacts. The paper also presents a detailed analysis of culling and depth approximation strategies, as well as pipeline optimizations and workload distribution schemes for the compute-mode 3DGS hierarchical renderer. It discusses various sorting strategies for Gaussian splats and their influence on overall rendering quality and view-consistency. Additionally, the paper presents an effective automatic method to detect popping artifacts in videos captured from trained 3D Gaussians and a user study confirming the results of the presented method. The results show that a full per-pixel sorted renderer for Gaussian splats eliminates all popping artifacts but reduces rendering speed by 100×. The hierarchical renderer is virtually indistinguishable from a full per-pixel sorted renderer, but only adds an overhead of 4% compared to the original 3DGS. The hierarchical approach reduces the number of Gaussians by approximately half with nearly identical quality and view-consistency, leading to nearly doubled rendering performance and a 50% reduction in memory requirements. The paper also discusses the performance and ablation studies of the proposed method, showing that it achieves competitive runtimes on all evaluated scenes. The method is able to handle large Gaussians and significantly reduces popping artifacts compared to 3DGS. The paper concludes that the proposed approach is view-consistent, reduces memory usage, and achieves virtually indistinguishable quality compared to 3DGS.StopThePop: Sorted Gaussian Splatting for View-Consistent Real-time Rendering This paper presents a novel hierarchical rasterization approach for 3D Gaussian Splatting (3DGS) to address popping artifacts and view inconsistencies during camera rotations. The main challenge in 3DGS is the approximate global sorting scheme that leads to popping artifacts during view rotation. The proposed method introduces a hierarchical per-pixel sorting strategy to effectively circumvent both short-range popping artifacts and long-range view inconsistencies. The paper discusses the limitations of 3DGS, which relies on a global sort based on the view-space z-coordinate of each Gaussian's mean. This approach leads to popping artifacts during camera rotations due to changes in the relative depth of shells. The proposed method improves rendering consistency by computing the optimal depth for each Gaussian along the view ray, which allows for accurate sorting and eliminates popping artifacts. The paper also presents a detailed analysis of culling and depth approximation strategies, as well as pipeline optimizations and workload distribution schemes for the compute-mode 3DGS hierarchical renderer. It discusses various sorting strategies for Gaussian splats and their influence on overall rendering quality and view-consistency. Additionally, the paper presents an effective automatic method to detect popping artifacts in videos captured from trained 3D Gaussians and a user study confirming the results of the presented method. The results show that a full per-pixel sorted renderer for Gaussian splats eliminates all popping artifacts but reduces rendering speed by 100×. The hierarchical renderer is virtually indistinguishable from a full per-pixel sorted renderer, but only adds an overhead of 4% compared to the original 3DGS. The hierarchical approach reduces the number of Gaussians by approximately half with nearly identical quality and view-consistency, leading to nearly doubled rendering performance and a 50% reduction in memory requirements. The paper also discusses the performance and ablation studies of the proposed method, showing that it achieves competitive runtimes on all evaluated scenes. The method is able to handle large Gaussians and significantly reduces popping artifacts compared to 3DGS. The paper concludes that the proposed approach is view-consistent, reduces memory usage, and achieves virtually indistinguishable quality compared to 3DGS.
Reach us at info@study.space