Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

8 Jul 2024 | Yuanhao Cai, Yixun Liang, Jiahao Wang, Angtian Wang, Yulun Zhang, Xiaokang Yang, Zongwei Zhou, and Alan Yuille
The paper introduces X-Gaussian, a novel 3D Gaussian splatting-based framework for X-ray novel view synthesis (NVS). Existing methods, primarily based on Neural Radiance Fields (NeRF), suffer from long training times and slow inference speeds. X-Gaussian addresses these issues by: 1. **Radiative Gaussian Point Cloud Model**: It redesigns a radiative Gaussian point cloud model inspired by the isotropic nature of X-ray imaging, excluding the influence of view direction when predicting radiation intensity. This model is more suitable for X-ray imaging compared to the spherical harmonics (SH) used in RGB 3D Gaussian splatting (3DGS). 2. **Differentiable Radiative Rasterization (DRR)**: Based on the radiative Gaussian point cloud model, X-Gaussian develops a GPU-friendly DRR with CUDA implementation, which is faster than traditional RGB rasterization. 3. **Angle-pose Cuboid Uniform Initialization (ACUI)**: It introduces an ACUI strategy that uses the parameters of the X-ray scanner to compute camera information and uniformly samples point positions within a cuboid enclosing the scanned object, significantly reducing training time. Experiments show that X-Gaussian outperforms state-of-the-art (SOTA) methods by 6.5 dB while achieving 73× faster inference speed and 7× shorter training time. The method also demonstrates practical value in sparse-view CT reconstruction, improving the quality of reconstructed images.The paper introduces X-Gaussian, a novel 3D Gaussian splatting-based framework for X-ray novel view synthesis (NVS). Existing methods, primarily based on Neural Radiance Fields (NeRF), suffer from long training times and slow inference speeds. X-Gaussian addresses these issues by: 1. **Radiative Gaussian Point Cloud Model**: It redesigns a radiative Gaussian point cloud model inspired by the isotropic nature of X-ray imaging, excluding the influence of view direction when predicting radiation intensity. This model is more suitable for X-ray imaging compared to the spherical harmonics (SH) used in RGB 3D Gaussian splatting (3DGS). 2. **Differentiable Radiative Rasterization (DRR)**: Based on the radiative Gaussian point cloud model, X-Gaussian develops a GPU-friendly DRR with CUDA implementation, which is faster than traditional RGB rasterization. 3. **Angle-pose Cuboid Uniform Initialization (ACUI)**: It introduces an ACUI strategy that uses the parameters of the X-ray scanner to compute camera information and uniformly samples point positions within a cuboid enclosing the scanned object, significantly reducing training time. Experiments show that X-Gaussian outperforms state-of-the-art (SOTA) methods by 6.5 dB while achieving 73× faster inference speed and 7× shorter training time. The method also demonstrates practical value in sparse-view CT reconstruction, improving the quality of reconstructed images.
Reach us at info@study.space
[slides] Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis | StudySpace