EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting

EndoGSLAM: Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting

22 Mar 2024 | Kailing Wang*, Chen Yang*, Yuehao Wang, Sikuang Li, Yan Wang, Qi Dou, Xiaokang Yang, Wei Shen
EndoGSLAM is an efficient SLAM approach for endoscopic surgeries that enables real-time dense reconstruction and tracking. It integrates a streamlined Gaussian representation and differentiable rasterization to achieve over 100 fps rendering speed during online camera tracking and tissue reconstruction. Extensive experiments show that EndoGSLAM achieves a better trade-off between intraoperative availability and reconstruction quality than traditional or neural SLAM approaches, demonstrating its potential for endoscopic surgeries. The method uses a simplified Gaussian representation and differentiable rasterization to facilitate fast optimization and rendering. Unlike traditional or implicit SLAM representations, EndoGSLAM can use dense photometric loss for real-time tracking and reconstruction, making it robust in complex surgical fields. Additionally, EndoGSLAM iteratively expands 3D Gaussians on previously unobserved regions and partially refines the reconstructed surgical field, significantly reducing computational costs. The method is evaluated on the C3VD dataset, showing superior performance in terms of optimization speed, rendering quality, and system efficiency. EndoGSLAM-H achieves high-quality reconstruction and real-time rendering, while EndoGSLAM-R prioritizes real-time processing at the expense of some performance. The method is compared to ORB-SLAM3, NICESLAM, and Endo-Depth, showing its advantages in real-time rendering and reconstruction. The ablation study shows that the pre-filter M, keyframe-based refining strategy, and Gaussian simplification contribute to the method's performance. The conclusion is that EndoGSLAM is an advanced dense SLAM framework that enables accurate localization, high-quality reconstruction, and real-time visualization, with potential for future work in eliminating depth reliance and integrating into surgical navigation systems.EndoGSLAM is an efficient SLAM approach for endoscopic surgeries that enables real-time dense reconstruction and tracking. It integrates a streamlined Gaussian representation and differentiable rasterization to achieve over 100 fps rendering speed during online camera tracking and tissue reconstruction. Extensive experiments show that EndoGSLAM achieves a better trade-off between intraoperative availability and reconstruction quality than traditional or neural SLAM approaches, demonstrating its potential for endoscopic surgeries. The method uses a simplified Gaussian representation and differentiable rasterization to facilitate fast optimization and rendering. Unlike traditional or implicit SLAM representations, EndoGSLAM can use dense photometric loss for real-time tracking and reconstruction, making it robust in complex surgical fields. Additionally, EndoGSLAM iteratively expands 3D Gaussians on previously unobserved regions and partially refines the reconstructed surgical field, significantly reducing computational costs. The method is evaluated on the C3VD dataset, showing superior performance in terms of optimization speed, rendering quality, and system efficiency. EndoGSLAM-H achieves high-quality reconstruction and real-time rendering, while EndoGSLAM-R prioritizes real-time processing at the expense of some performance. The method is compared to ORB-SLAM3, NICESLAM, and Endo-Depth, showing its advantages in real-time rendering and reconstruction. The ablation study shows that the pre-filter M, keyframe-based refining strategy, and Gaussian simplification contribute to the method's performance. The conclusion is that EndoGSLAM is an advanced dense SLAM framework that enables accurate localization, high-quality reconstruction, and real-time visualization, with potential for future work in eliminating depth reliance and integrating into surgical navigation systems.
Reach us at info@study.space
Understanding EndoGSLAM%3A Real-Time Dense Reconstruction and Tracking in Endoscopic Surgeries using Gaussian Splatting