CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians This paper proposes a method for sparse novel view synthesis using 3D Gaussian Splatting (3DGS). The key idea is to introduce coherence to the 3D Gaussians during optimization. This is achieved by assigning a single Gaussian to each pixel in every input image and enforcing single and multiview constraints in 2D image space. The method also introduces depth-based initialization to position the Gaussians in the world space. The approach is evaluated on various scenes and shows significant improvements compared to state-of-the-art sparse-view NeRF-based approaches. The method addresses the issue of overfitting in 3DGS when dealing with sparse input images. By introducing coherence, the Gaussians are constrained to move in a structured manner, preventing them from independently changing during optimization. This is done through an implicit convolutional decoder and a total variation loss. Additionally, a flow-based loss function is introduced to ensure the positions of Gaussians corresponding to the same pixels in different images are similar. The method also uses monocular depth estimates to initialize the Gaussians, which helps in positioning them correctly in the world space. This initialization is complemented by regularized optimization that encourages the updates to be coherent and smooth. This allows for high-quality texture and geometry reconstruction. The method is evaluated on the LLFF and NVS-RGBD datasets, showing superior performance in terms of PSNR, SSIM, and LPIPS. The results demonstrate that the method can reconstruct high-quality texture and geometry, even in the presence of occluded regions. The method also allows for the identification and inpainting of occluded regions, producing realistic hallucinated details. The method introduces a novel approach to regularize the 3DGS optimization for sparse input settings. It uses a combination of an implicit decoder and total variation loss to enforce coherence in the Gaussians. The method also uses flow-based regularization to ensure that corresponding points in different images are consistently positioned. This results in a more coherent and accurate reconstruction of the 3D scene.CoherentGS: Sparse Novel View Synthesis with Coherent 3D Gaussians This paper proposes a method for sparse novel view synthesis using 3D Gaussian Splatting (3DGS). The key idea is to introduce coherence to the 3D Gaussians during optimization. This is achieved by assigning a single Gaussian to each pixel in every input image and enforcing single and multiview constraints in 2D image space. The method also introduces depth-based initialization to position the Gaussians in the world space. The approach is evaluated on various scenes and shows significant improvements compared to state-of-the-art sparse-view NeRF-based approaches. The method addresses the issue of overfitting in 3DGS when dealing with sparse input images. By introducing coherence, the Gaussians are constrained to move in a structured manner, preventing them from independently changing during optimization. This is done through an implicit convolutional decoder and a total variation loss. Additionally, a flow-based loss function is introduced to ensure the positions of Gaussians corresponding to the same pixels in different images are similar. The method also uses monocular depth estimates to initialize the Gaussians, which helps in positioning them correctly in the world space. This initialization is complemented by regularized optimization that encourages the updates to be coherent and smooth. This allows for high-quality texture and geometry reconstruction. The method is evaluated on the LLFF and NVS-RGBD datasets, showing superior performance in terms of PSNR, SSIM, and LPIPS. The results demonstrate that the method can reconstruct high-quality texture and geometry, even in the presence of occluded regions. The method also allows for the identification and inpainting of occluded regions, producing realistic hallucinated details. The method introduces a novel approach to regularize the 3DGS optimization for sparse input settings. It uses a combination of an implicit decoder and total variation loss to enforce coherence in the Gaussians. The method also uses flow-based regularization to ensure that corresponding points in different images are consistently positioned. This results in a more coherent and accurate reconstruction of the 3D scene.