2024 | Zhicheng Lu, Xiang Guo, Le Hui, Tianrui Chen, Min Yang, Xiao Tang, Feng Zhu, Yuchao Dai
This paper proposes a 3D geometry-aware deformable Gaussian Splatting method for dynamic view synthesis. Existing neural radiance fields (NeRF) based solutions learn deformation in an implicit manner, which cannot incorporate 3D scene geometry, leading to unsatisfactory dynamic view synthesis and 3D dynamic reconstruction. The proposed method uses 3D Gaussian Splatting to represent scenes as a collection of 3D Gaussians, where each Gaussian is optimized to move and rotate over time to model deformation. To enforce 3D scene geometry constraints during deformation, the method explicitly extracts 3D geometry features and integrates them in learning the 3D deformation. This enables improved dynamic view synthesis and 3D dynamic reconstruction. Extensive experiments on both synthetic and real datasets show that the method achieves new state-of-the-art performance. The method consists of a Gaussian canonical field and a deformation field. The Gaussian canonical field consists of 3D Gaussian distributions and a geometry-aware feature learning network. The deformation field estimates a transformation for each Gaussian in the canonical field, which transfers the Gaussian from the canonical field to the given timestamp. The method uses 3D Gaussian splatting to render images for the given timestamp. The main contributions are: (1) a geometry-aware feature extraction network based on 3D Gaussian distribution to better utilize local geometric information; (2) use of continuous 6D rotation representation and modified density control strategy to adapt Gaussian splatting to dynamic scenes; and (3) extensive experiments on both synthetic and real datasets show that the method surpasses competing methods by a wide margin. The method is evaluated on synthetic and real datasets, including D-NeRF and HyperNeRF. The results show that the method achieves high-quality rendering and accurate 3D reconstruction. The method is efficient and can handle dynamic scenes with complex deformations. The method is also compared with other state-of-the-art methods, and it outperforms them in terms of PSNR, SSIM, and LPIPS. The method is also visualized, and the results show that the method can recover accurate and detailed images. The method is also compared with other methods in terms of ablation studies, and it shows that the geometry-aware features and 6D rotation representation are important for the performance. The method is also compared with other methods in terms of density control, and it shows that the method can adaptively control the density of Gaussians. The method is also compared with other methods in terms of temporal interpolation, and it shows that the method can interpolate between different time steps. The method is also compared with other methods in terms of motion modeling, and it shows that the method can model dynamic scenes with complex deformations. The method is also compared with other methods in terms of computational efficiency, and it shows that the method is efficient and can handle dynamic scenes with complex deformations. The method is also compared with other methods in terms of generalization ability,This paper proposes a 3D geometry-aware deformable Gaussian Splatting method for dynamic view synthesis. Existing neural radiance fields (NeRF) based solutions learn deformation in an implicit manner, which cannot incorporate 3D scene geometry, leading to unsatisfactory dynamic view synthesis and 3D dynamic reconstruction. The proposed method uses 3D Gaussian Splatting to represent scenes as a collection of 3D Gaussians, where each Gaussian is optimized to move and rotate over time to model deformation. To enforce 3D scene geometry constraints during deformation, the method explicitly extracts 3D geometry features and integrates them in learning the 3D deformation. This enables improved dynamic view synthesis and 3D dynamic reconstruction. Extensive experiments on both synthetic and real datasets show that the method achieves new state-of-the-art performance. The method consists of a Gaussian canonical field and a deformation field. The Gaussian canonical field consists of 3D Gaussian distributions and a geometry-aware feature learning network. The deformation field estimates a transformation for each Gaussian in the canonical field, which transfers the Gaussian from the canonical field to the given timestamp. The method uses 3D Gaussian splatting to render images for the given timestamp. The main contributions are: (1) a geometry-aware feature extraction network based on 3D Gaussian distribution to better utilize local geometric information; (2) use of continuous 6D rotation representation and modified density control strategy to adapt Gaussian splatting to dynamic scenes; and (3) extensive experiments on both synthetic and real datasets show that the method surpasses competing methods by a wide margin. The method is evaluated on synthetic and real datasets, including D-NeRF and HyperNeRF. The results show that the method achieves high-quality rendering and accurate 3D reconstruction. The method is efficient and can handle dynamic scenes with complex deformations. The method is also compared with other state-of-the-art methods, and it outperforms them in terms of PSNR, SSIM, and LPIPS. The method is also visualized, and the results show that the method can recover accurate and detailed images. The method is also compared with other methods in terms of ablation studies, and it shows that the geometry-aware features and 6D rotation representation are important for the performance. The method is also compared with other methods in terms of density control, and it shows that the method can adaptively control the density of Gaussians. The method is also compared with other methods in terms of temporal interpolation, and it shows that the method can interpolate between different time steps. The method is also compared with other methods in terms of motion modeling, and it shows that the method can model dynamic scenes with complex deformations. The method is also compared with other methods in terms of computational efficiency, and it shows that the method is efficient and can handle dynamic scenes with complex deformations. The method is also compared with other methods in terms of generalization ability,