This paper addresses the challenge of accurately reconstructing complex dynamic scenes using deformable 3D Gaussian Splatting (3DGS). Traditional methods often struggle with this task due to their reliance on coordinate-based deformation fields, which are inadequate for representing the mixture of multiple fields centered at Gaussians. To overcome this, the authors propose a novel approach that uses per-Gaussian latent embeddings to predict deformations for each Gaussian, improving the clarity and detail of the reconstructed scenes.
The key contributions of the paper include:
1. **Per-Gaussian Embeddings**: Each Gaussian is assigned a latent embedding, which is used to predict deformations. This allows for precise modeling of different deformations of individual Gaussians.
2. **Coarse-Fine Deformation**: The temporal variations of Gaussian parameters are decomposed into coarse and fine components. Coarse deformation handles large or slow movements, while fine deformation captures fast or detailed movements.
3. **Local Smoothness Regularization**: This regularization ensures that neighboring Gaussians have similar deformations, enhancing the details in dynamic regions.
The authors evaluate their method on several datasets, including the Neural 3D Video dataset, Technicolor Light Field dataset, and HyperNeRF dataset. Their method outperforms existing baselines in terms of reconstruction quality, frame rate, and model size. The paper also includes a detailed analysis and ablation study to demonstrate the effectiveness of each component of the proposed method.
Overall, the paper provides a robust and efficient solution for deformable 3D Gaussian Splatting, making it suitable for applications such as dynamic scene reconstruction, content production, and mixed reality.This paper addresses the challenge of accurately reconstructing complex dynamic scenes using deformable 3D Gaussian Splatting (3DGS). Traditional methods often struggle with this task due to their reliance on coordinate-based deformation fields, which are inadequate for representing the mixture of multiple fields centered at Gaussians. To overcome this, the authors propose a novel approach that uses per-Gaussian latent embeddings to predict deformations for each Gaussian, improving the clarity and detail of the reconstructed scenes.
The key contributions of the paper include:
1. **Per-Gaussian Embeddings**: Each Gaussian is assigned a latent embedding, which is used to predict deformations. This allows for precise modeling of different deformations of individual Gaussians.
2. **Coarse-Fine Deformation**: The temporal variations of Gaussian parameters are decomposed into coarse and fine components. Coarse deformation handles large or slow movements, while fine deformation captures fast or detailed movements.
3. **Local Smoothness Regularization**: This regularization ensures that neighboring Gaussians have similar deformations, enhancing the details in dynamic regions.
The authors evaluate their method on several datasets, including the Neural 3D Video dataset, Technicolor Light Field dataset, and HyperNeRF dataset. Their method outperforms existing baselines in terms of reconstruction quality, frame rate, and model size. The paper also includes a detailed analysis and ablation study to demonstrate the effectiveness of each component of the proposed method.
Overall, the paper provides a robust and efficient solution for deformable 3D Gaussian Splatting, making it suitable for applications such as dynamic scene reconstruction, content production, and mixed reality.