Score Distillation via Reparametrized DDIM

Score Distillation via Reparametrized DDIM

13 Jun 2024 | Artem Lukoianov, Haitz Sáez de Ocáriz Borde, Kristjan Greenwald, Vitor Campagnolo Guizilini, Timur Bagautdinov, Vincent Sitzmann, Justin Solomon
This paper proposes a novel method called Score Distillation via Inversion (SDI) to improve 3D generation using diffusion models. The authors analyze the Score Distillation Sampling (SDS) algorithm and find that it can be understood as an approximation of the Denoising Diffusion Implicit Model (DDIM) velocity field. However, SDS introduces excessive variance due to its random noise sampling, leading to over-smoothed and unrealistic 3D shapes. To address this, the authors propose SDI, which replaces the random noise sampling in SDS with prompt-conditioned DDIM inversion. This modification significantly improves 3D generation quality, making it almost identical to DDIM in 2D and significantly better in 3D. The method achieves better or similar 3D generation quality compared to other state-of-the-art Score Distillation methods without training additional neural networks or multi-view supervision. The authors also show that SDI produces high-quality textures consistent with the 2D diffusion model and improves geometric and temporal coherency for 3D generation. The method is evaluated on various 3D generation tasks and shows promising results. The key contributions of the paper include the derivation of SDS as a reparameterization of DDIM, the proposal of SDI as an improved method for 3D generation, and the systematic comparison of SDI with other state-of-the-art methods. The paper also discusses the limitations of the method and potential future work.This paper proposes a novel method called Score Distillation via Inversion (SDI) to improve 3D generation using diffusion models. The authors analyze the Score Distillation Sampling (SDS) algorithm and find that it can be understood as an approximation of the Denoising Diffusion Implicit Model (DDIM) velocity field. However, SDS introduces excessive variance due to its random noise sampling, leading to over-smoothed and unrealistic 3D shapes. To address this, the authors propose SDI, which replaces the random noise sampling in SDS with prompt-conditioned DDIM inversion. This modification significantly improves 3D generation quality, making it almost identical to DDIM in 2D and significantly better in 3D. The method achieves better or similar 3D generation quality compared to other state-of-the-art Score Distillation methods without training additional neural networks or multi-view supervision. The authors also show that SDI produces high-quality textures consistent with the 2D diffusion model and improves geometric and temporal coherency for 3D generation. The method is evaluated on various 3D generation tasks and shows promising results. The key contributions of the paper include the derivation of SDS as a reparameterization of DDIM, the proposal of SDI as an improved method for 3D generation, and the systematic comparison of SDI with other state-of-the-art methods. The paper also discusses the limitations of the method and potential future work.
Reach us at info@study.space
Understanding Score Distillation via Reparametrized DDIM