Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis

8 Jul 2024 | Yuanhao Cai, Yixun Liang, Jiahao Wang, Angtian Wang, Yulun Zhang, Xiaokang Yang, Zongwei Zhou, and Alan Yuille
This paper proposes X-Gaussian, a 3D Gaussian splatting-based framework for X-ray novel view synthesis. X-Gaussian addresses the limitations of existing methods, which are based on Neural Radiance Fields (NeRF) and suffer from long training times and slow inference speeds. The proposed framework is designed to efficiently synthesize X-ray projections from new viewpoints, leveraging the isotropic nature of X-ray imaging. X-Gaussian introduces two key techniques: a radiative Gaussian point cloud model and a Differentiable Radiative Rasterization (DRR) method. The radiative Gaussian point cloud model replaces the spherical harmonics (SH) used in RGB-based 3D Gaussian splatting with a Radiation Intensity Response Function (RIRF), which models the isotropic radiation intensity of 3D points. This model excludes the influence of view direction, making it suitable for X-ray imaging. The DRR method enables efficient rendering of X-ray projections using CUDA, significantly improving inference speed. Additionally, X-Gaussian employs an Angle-pose Cuboid Uniform Initialization (ACUI) strategy to initialize the Gaussian point clouds. This strategy avoids the need for time-consuming Structure-from-Motion (SfM) algorithms by directly using the parameters of the X-ray scanner to compute camera matrices and uniformly sampling point positions within a cuboid enclosing the scanned object. Experiments show that X-Gaussian outperforms state-of-the-art methods by 6.5 dB in PSNR while achieving 73× faster inference speed and less than 15% training time. The method also demonstrates practical value in sparse-view CT reconstruction, where it significantly improves reconstruction quality. The framework is efficient, scalable, and suitable for real-time applications in medical imaging.This paper proposes X-Gaussian, a 3D Gaussian splatting-based framework for X-ray novel view synthesis. X-Gaussian addresses the limitations of existing methods, which are based on Neural Radiance Fields (NeRF) and suffer from long training times and slow inference speeds. The proposed framework is designed to efficiently synthesize X-ray projections from new viewpoints, leveraging the isotropic nature of X-ray imaging. X-Gaussian introduces two key techniques: a radiative Gaussian point cloud model and a Differentiable Radiative Rasterization (DRR) method. The radiative Gaussian point cloud model replaces the spherical harmonics (SH) used in RGB-based 3D Gaussian splatting with a Radiation Intensity Response Function (RIRF), which models the isotropic radiation intensity of 3D points. This model excludes the influence of view direction, making it suitable for X-ray imaging. The DRR method enables efficient rendering of X-ray projections using CUDA, significantly improving inference speed. Additionally, X-Gaussian employs an Angle-pose Cuboid Uniform Initialization (ACUI) strategy to initialize the Gaussian point clouds. This strategy avoids the need for time-consuming Structure-from-Motion (SfM) algorithms by directly using the parameters of the X-ray scanner to compute camera matrices and uniformly sampling point positions within a cuboid enclosing the scanned object. Experiments show that X-Gaussian outperforms state-of-the-art methods by 6.5 dB in PSNR while achieving 73× faster inference speed and less than 15% training time. The method also demonstrates practical value in sparse-view CT reconstruction, where it significantly improves reconstruction quality. The framework is efficient, scalable, and suitable for real-time applications in medical imaging.
Reach us at info@study.space
[slides] Radiative Gaussian Splatting for Efficient X-ray Novel View Synthesis | StudySpace