Endo-4DGS is a real-time endoscopic scene reconstruction method that uses 4D Gaussian Splatting (GS) for 3D representation. The method addresses the challenges of dynamic scene reconstruction in minimally invasive surgery, where traditional techniques struggle due to limited field-of-view, occlusions, and dynamic tissue deformation. Endo-4DGS leverages a lightweight MLP to capture temporal dynamics with Gaussian deformation fields and uses a powerful depth estimation model, Depth-Anything, to generate pseudo-depth maps for Gaussian initialization. It also incorporates confidence-guided learning to tackle ill-posed problems in monocular depth estimation and enhances depth-guided reconstruction with surface normal constraints and depth regularization. The method has been validated on two surgical datasets, demonstrating effective real-time rendering, efficient computation, and accurate reconstruction. The code is available at https://github.com/lastbasket/Endo-4DGS. The method introduces 4D Gaussian Splatting for deformable scene representation, incorporating the temporal dimension as the fourth axis to model dynamic environments. It utilizes a depth prior-based reconstruction approach, which relies on multi-view information and static scene assumptions, which are not always feasible in surgical scenarios. To address these challenges, the method employs Depth-Anything, a cutting-edge method trained through extensive visual pre-training, to project pre-trained depth into 3D for robust 4D Gaussian initialization. Confidence-guided learning is introduced to reduce the influence of noisy or uncertain measurements in the pre-trained depth estimation. Surface normal constraints and depth regularization are also implemented to strengthen the pseudo-depth's accuracy and geometry constraint. The method's representation and rendering formula for 4D Gaussians are detailed, including the use of a multi-head Gaussian deformation decoder and a spatial-temporal encoder. The method's performance is evaluated on two publicly available datasets, StereoMIS and EndoNeRF, showing superior performance in terms of 3D scene reconstruction, real-time inference speed, and reduced training time and GPU memory usage. The results demonstrate that Endo-4DGS outperforms existing methods in both datasets, achieving a real-time inference speed of 100 FPS with only 4 minutes of training and 4GB of GPU memory. The method's effectiveness is supported by both quantitative and qualitative results, highlighting its potential for future real-time endoscopic applications.Endo-4DGS is a real-time endoscopic scene reconstruction method that uses 4D Gaussian Splatting (GS) for 3D representation. The method addresses the challenges of dynamic scene reconstruction in minimally invasive surgery, where traditional techniques struggle due to limited field-of-view, occlusions, and dynamic tissue deformation. Endo-4DGS leverages a lightweight MLP to capture temporal dynamics with Gaussian deformation fields and uses a powerful depth estimation model, Depth-Anything, to generate pseudo-depth maps for Gaussian initialization. It also incorporates confidence-guided learning to tackle ill-posed problems in monocular depth estimation and enhances depth-guided reconstruction with surface normal constraints and depth regularization. The method has been validated on two surgical datasets, demonstrating effective real-time rendering, efficient computation, and accurate reconstruction. The code is available at https://github.com/lastbasket/Endo-4DGS. The method introduces 4D Gaussian Splatting for deformable scene representation, incorporating the temporal dimension as the fourth axis to model dynamic environments. It utilizes a depth prior-based reconstruction approach, which relies on multi-view information and static scene assumptions, which are not always feasible in surgical scenarios. To address these challenges, the method employs Depth-Anything, a cutting-edge method trained through extensive visual pre-training, to project pre-trained depth into 3D for robust 4D Gaussian initialization. Confidence-guided learning is introduced to reduce the influence of noisy or uncertain measurements in the pre-trained depth estimation. Surface normal constraints and depth regularization are also implemented to strengthen the pseudo-depth's accuracy and geometry constraint. The method's representation and rendering formula for 4D Gaussians are detailed, including the use of a multi-head Gaussian deformation decoder and a spatial-temporal encoder. The method's performance is evaluated on two publicly available datasets, StereoMIS and EndoNeRF, showing superior performance in terms of 3D scene reconstruction, real-time inference speed, and reduced training time and GPU memory usage. The results demonstrate that Endo-4DGS outperforms existing methods in both datasets, achieving a real-time inference speed of 100 FPS with only 4 minutes of training and 4GB of GPU memory. The method's effectiveness is supported by both quantitative and qualitative results, highlighting its potential for future real-time endoscopic applications.