End-to-End Rate-Distortion Optimized 3D Gaussian Representation

End-to-End Rate-Distortion Optimized 3D Gaussian Representation

9 Apr 2024 | Henan Wang1, Hanxin Zhu1, Tianyu He2, Runsen Feng1, Jiajun Deng3, Jiang Bian2, and Zhibo Chen1
The paper "End-to-End Rate-Distortion Optimized 3D Gaussian Representation" addresses the significant storage overhead of 3D Gaussian Splatting (3DGS), a technique used for 3D representation and image rendering. The authors propose RDO-Gaussian, an end-to-end Rate-Distortion Optimization (RDO) framework that achieves flexible and continuous rate control. RDO-Gaussian introduces dynamic pruning and entropy-constrained vector quantization (ECVQ) to optimize both rate and distortion simultaneously. It also models the colors of different regions and materials with learnable parameters, addressing the issue of treating all colors equally in previous methods. The method is evaluated on both real and synthetic scenes, showing a reduction in 3D Gaussian size by over 40× while maintaining or improving rendering quality. The main contributions include formulating 3D Gaussian representation learning as a joint optimization of rate and distortion, achieving volumetric bit allocation through adaptive spherical harmonics pruning, and surpassing existing methods in rate-distortion performance. The source code is available at https://github.com/USTC-IMCL/RDO-Gaussian.The paper "End-to-End Rate-Distortion Optimized 3D Gaussian Representation" addresses the significant storage overhead of 3D Gaussian Splatting (3DGS), a technique used for 3D representation and image rendering. The authors propose RDO-Gaussian, an end-to-end Rate-Distortion Optimization (RDO) framework that achieves flexible and continuous rate control. RDO-Gaussian introduces dynamic pruning and entropy-constrained vector quantization (ECVQ) to optimize both rate and distortion simultaneously. It also models the colors of different regions and materials with learnable parameters, addressing the issue of treating all colors equally in previous methods. The method is evaluated on both real and synthetic scenes, showing a reduction in 3D Gaussian size by over 40× while maintaining or improving rendering quality. The main contributions include formulating 3D Gaussian representation learning as a joint optimization of rate and distortion, achieving volumetric bit allocation through adaptive spherical harmonics pruning, and surpassing existing methods in rate-distortion performance. The source code is available at https://github.com/USTC-IMCL/RDO-Gaussian.
Reach us at info@study.space
[slides and audio] End-to-End Rate-Distortion Optimized 3D Gaussian Representation