EndoGS is a method for deformable endoscopic tissue reconstruction using Gaussian Splatting. It addresses the challenges of reconstructing high-quality 3D deformable tissues from single-viewpoint videos, estimated depth maps, and labeled tool masks. The method incorporates deformation fields to handle dynamic scenes, depth-guided supervision with spatial-temporal weight masks to optimize 3D targets with tool occlusion, and surface-aligned regularization terms to capture better geometry. EndoGS reconstructs and renders high-quality deformable endoscopic tissues from a single-viewpoint video, estimated depth maps, and labeled tool masks. Experiments on DaVinci robotic surgery videos demonstrate that EndoGS achieves superior rendering quality. Code is available at https://github.com/HKU-MedAI/EndoGS.
The main contributions of EndoGS are: 1) It presents the first Gaussian Splatting-based method for deformable endoscopic tissues reconstruction. 2) It represents dynamic surgical scenes with static Gaussians and deformable parameters in the time dimension, adopts depth-guided supervision with spatial-temporal weight masks for monocular optimization with tool occlusion, and combines surface-aligned regularization terms to capture better geometry. 3) It uses the same input tool masks involved in the training and inference for comparison methods and makes a clear and fair comparison, and experiments demonstrate its superior performance.
EndoGS utilizes a deformable variant of 3D-GS to reconstruct 3D surgical scenes from a single-viewpoint video, estimated depth maps, and labeled tool masks. It combines extracted binary tool masks and depth maps estimated from binocular captures for the left views. The method uses a lightweight MLP to represent the dynamic field and merges all information to decode the deformation of the position, scaling factor, rotation factor, spherical harmonic coefficients, and opacity. It also uses depth-guided loss with estimated depth maps and total variation losses in the spatial and temporal dimensions to serve as additional regularization. Surface-aligned Gaussians are used to ensure tight integration with the surface, and surface-aligned regularization is applied to improve the quality of the reconstruction.
Experiments on DaVinci robotic surgery videos demonstrate that EndoGS achieves superior rendering quality and real-time rendering speeds. It outperforms other methods in terms of rendering quality and speed, and is able to handle tool occlusion and dynamic deformations effectively. The method is evaluated using standard image quality metrics, including PSNR, SSIM, and LPIPS, and the results show that EndoGS achieves superior performance compared to other methods. The method is also compared with other methods in terms of rendering quality and speed, and the results show that EndoGS achieves superior performance. The method is able to handle tool occlusion and dynamic deformations effectively, and is able to achieve real-time rendering speeds.EndoGS is a method for deformable endoscopic tissue reconstruction using Gaussian Splatting. It addresses the challenges of reconstructing high-quality 3D deformable tissues from single-viewpoint videos, estimated depth maps, and labeled tool masks. The method incorporates deformation fields to handle dynamic scenes, depth-guided supervision with spatial-temporal weight masks to optimize 3D targets with tool occlusion, and surface-aligned regularization terms to capture better geometry. EndoGS reconstructs and renders high-quality deformable endoscopic tissues from a single-viewpoint video, estimated depth maps, and labeled tool masks. Experiments on DaVinci robotic surgery videos demonstrate that EndoGS achieves superior rendering quality. Code is available at https://github.com/HKU-MedAI/EndoGS.
The main contributions of EndoGS are: 1) It presents the first Gaussian Splatting-based method for deformable endoscopic tissues reconstruction. 2) It represents dynamic surgical scenes with static Gaussians and deformable parameters in the time dimension, adopts depth-guided supervision with spatial-temporal weight masks for monocular optimization with tool occlusion, and combines surface-aligned regularization terms to capture better geometry. 3) It uses the same input tool masks involved in the training and inference for comparison methods and makes a clear and fair comparison, and experiments demonstrate its superior performance.
EndoGS utilizes a deformable variant of 3D-GS to reconstruct 3D surgical scenes from a single-viewpoint video, estimated depth maps, and labeled tool masks. It combines extracted binary tool masks and depth maps estimated from binocular captures for the left views. The method uses a lightweight MLP to represent the dynamic field and merges all information to decode the deformation of the position, scaling factor, rotation factor, spherical harmonic coefficients, and opacity. It also uses depth-guided loss with estimated depth maps and total variation losses in the spatial and temporal dimensions to serve as additional regularization. Surface-aligned Gaussians are used to ensure tight integration with the surface, and surface-aligned regularization is applied to improve the quality of the reconstruction.
Experiments on DaVinci robotic surgery videos demonstrate that EndoGS achieves superior rendering quality and real-time rendering speeds. It outperforms other methods in terms of rendering quality and speed, and is able to handle tool occlusion and dynamic deformations effectively. The method is evaluated using standard image quality metrics, including PSNR, SSIM, and LPIPS, and the results show that EndoGS achieves superior performance compared to other methods. The method is also compared with other methods in terms of rendering quality and speed, and the results show that EndoGS achieves superior performance. The method is able to handle tool occlusion and dynamic deformations effectively, and is able to achieve real-time rendering speeds.