SUNDAE: Spectrally Pruned Gaussian Fields with Neural Compensation
SUNDAE is a memory-efficient Gaussian field that combines spectral pruning and neural compensation to reduce storage while maintaining high rendering quality and speed. The method addresses the high memory consumption of 3D Gaussian Splatting (3DGS) by modeling the relationships between Gaussian primitives using graph signal processing and a neural compensation module. The graph-based pruning technique removes less important primitives while preserving essential information, and the neural compensation module restores rendering quality lost due to pruning. SUNDAE achieves high-quality rendering with significantly reduced memory usage, as demonstrated by results on the Mip-NeRF360 dataset, where it achieves 26.80 PSNR at 145 FPS using only 104 MB of memory, compared to 25.60 PSNR at 160 FPS using 523 MB for the vanilla 3DGS algorithm. The method is evaluated on three datasets and shows superior performance in terms of both quality and speed with low memory footprint. The contributions include a novel primitive pruning framework based on the spectrum of primitive graphs, a new feature splatting and mixing module to compensate for performance drops, and state-of-the-art results on various benchmarks. The method is implemented in Python using PyTorch and CUDA, with training and testing conducted on an NVIDIA RTX 3090 GPU. The results show that SUNDAE effectively reduces the size of Gaussian fields while preserving good quality and fast rendering speed.SUNDAE: Spectrally Pruned Gaussian Fields with Neural Compensation
SUNDAE is a memory-efficient Gaussian field that combines spectral pruning and neural compensation to reduce storage while maintaining high rendering quality and speed. The method addresses the high memory consumption of 3D Gaussian Splatting (3DGS) by modeling the relationships between Gaussian primitives using graph signal processing and a neural compensation module. The graph-based pruning technique removes less important primitives while preserving essential information, and the neural compensation module restores rendering quality lost due to pruning. SUNDAE achieves high-quality rendering with significantly reduced memory usage, as demonstrated by results on the Mip-NeRF360 dataset, where it achieves 26.80 PSNR at 145 FPS using only 104 MB of memory, compared to 25.60 PSNR at 160 FPS using 523 MB for the vanilla 3DGS algorithm. The method is evaluated on three datasets and shows superior performance in terms of both quality and speed with low memory footprint. The contributions include a novel primitive pruning framework based on the spectrum of primitive graphs, a new feature splatting and mixing module to compensate for performance drops, and state-of-the-art results on various benchmarks. The method is implemented in Python using PyTorch and CUDA, with training and testing conducted on an NVIDIA RTX 3090 GPU. The results show that SUNDAE effectively reduces the size of Gaussian fields while preserving good quality and fast rendering speed.