Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos

Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos

26 Jun 2024 | COLTON STEARNS, Stanford University, USA; ADAM HARLEY, Stanford University, USA; MIKAELA UY, Stanford University, USA; FLORIAN DUBOST, Google, USA; FEDERICO TOMBARI, Google, Switzerland; GORDON WETZSTEIN, Stanford University, USA; LEONIDAS GUIBAS, Stanford University, USA
The paper introduces Dynamic Gaussian Marbles (DGMarbles), a novel method for synthesizing novel views from casual monocular videos. DGMarbles addresses the challenge of underconstrained monocular settings by using isotropic Gaussian "marbles," a hierarchical divide-and-conquer learning strategy, and image and geometry priors. The method effectively captures 3D motion and scene geometry, achieving high-quality novel-view synthesis. Evaluations on the Nvidia Dynamic Scenes dataset and the Dycheck iPhone dataset show that DGMarbles outperforms other Gaussian baselines in quality and is comparable to non-Gaussian representations, while maintaining efficient rendering, tracking, and editability. The paper also discusses the limitations and future directions for open-world dynamic and monocular novel-view synthesis.The paper introduces Dynamic Gaussian Marbles (DGMarbles), a novel method for synthesizing novel views from casual monocular videos. DGMarbles addresses the challenge of underconstrained monocular settings by using isotropic Gaussian "marbles," a hierarchical divide-and-conquer learning strategy, and image and geometry priors. The method effectively captures 3D motion and scene geometry, achieving high-quality novel-view synthesis. Evaluations on the Nvidia Dynamic Scenes dataset and the Dycheck iPhone dataset show that DGMarbles outperforms other Gaussian baselines in quality and is comparable to non-Gaussian representations, while maintaining efficient rendering, tracking, and editability. The paper also discusses the limitations and future directions for open-world dynamic and monocular novel-view synthesis.
Reach us at info@study.space
[slides] Dynamic Gaussian Marbles for Novel View Synthesis of Casual Monocular Videos | StudySpace