GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting

GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting

15 Feb 2024 | Joanna Waczyńska, Piotr Borycki, Sławomir Tadeja, Jacek Tabor, Przemysław Spurek
GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting introduces a novel approach to Gaussian Splatting (GS) by integrating mesh-based parameterization. This method allows for real-time modification and adaptation of Gaussian components, enabling efficient rendering and editing of 3D scenes. Unlike traditional GS, which requires conditioning numerous Gaussian components, GaMeS uses a mesh to parameterize Gaussian components, allowing for automatic adjustments in position, scale, and rotation during animation. The model uses a pseudo-mesh, estimated during training, to modify GS without requiring a pre-existing mesh. This approach enables dynamic scene rendering and editing, with Gaussian components automatically adapting to mesh changes. GaMeS achieves this by parameterizing Gaussian components using mesh vertices, allowing for efficient training and real-time rendering. The method is demonstrated on various datasets, including synthetic and human face datasets, showing comparable results to existing methods. Experiments show that GaMeS can handle both static and animated scenes, with real-time modifications and high-quality rendering. The model's efficiency and flexibility make it suitable for applications requiring dynamic 3D scene modeling and editing. Limitations include potential artifacts with large mesh faces, which can be mitigated by subdividing them. Overall, GaMeS offers a practical solution for real-time 3D scene rendering and modification.GaMeS: Mesh-Based Adapting and Modification of Gaussian Splatting introduces a novel approach to Gaussian Splatting (GS) by integrating mesh-based parameterization. This method allows for real-time modification and adaptation of Gaussian components, enabling efficient rendering and editing of 3D scenes. Unlike traditional GS, which requires conditioning numerous Gaussian components, GaMeS uses a mesh to parameterize Gaussian components, allowing for automatic adjustments in position, scale, and rotation during animation. The model uses a pseudo-mesh, estimated during training, to modify GS without requiring a pre-existing mesh. This approach enables dynamic scene rendering and editing, with Gaussian components automatically adapting to mesh changes. GaMeS achieves this by parameterizing Gaussian components using mesh vertices, allowing for efficient training and real-time rendering. The method is demonstrated on various datasets, including synthetic and human face datasets, showing comparable results to existing methods. Experiments show that GaMeS can handle both static and animated scenes, with real-time modifications and high-quality rendering. The model's efficiency and flexibility make it suitable for applications requiring dynamic 3D scene modeling and editing. Limitations include potential artifacts with large mesh faces, which can be mitigated by subdividing them. Overall, GaMeS offers a practical solution for real-time 3D scene rendering and modification.
Reach us at info@study.space