22 Jul 2024 | Minghao Chen, Iro Laina, and Andrea Vedaldi
The paper introduces the Direct Gaussian Editor (DGE), a method for editing 3D objects and scenes based on open-ended language instructions. DGE addresses the inefficiencies and slow convergence issues of existing approaches, which often rely on iterative updates of costly 3D representations like neural radiance fields. DGE achieves this by modifying a high-quality image editor, such as InstructPix2Pix, to be multi-view consistent. This is done through a training-free approach that integrates cues from the 3D geometry of the scene. Given the multi-view consistent edited sequence of images, DGE directly and efficiently optimizes the 3D representation using 3D Gaussian Splatting. This method avoids the need for iterative updates, making it significantly more accurate and efficient compared to existing approaches. DGE also enables selective editing of specific parts of the scene, a capability not available in implicit radiance field representations. The paper includes a detailed description of the method, experimental results, and comparisons with other state-of-the-art methods, demonstrating the effectiveness and efficiency of DGE in 3D editing tasks.The paper introduces the Direct Gaussian Editor (DGE), a method for editing 3D objects and scenes based on open-ended language instructions. DGE addresses the inefficiencies and slow convergence issues of existing approaches, which often rely on iterative updates of costly 3D representations like neural radiance fields. DGE achieves this by modifying a high-quality image editor, such as InstructPix2Pix, to be multi-view consistent. This is done through a training-free approach that integrates cues from the 3D geometry of the scene. Given the multi-view consistent edited sequence of images, DGE directly and efficiently optimizes the 3D representation using 3D Gaussian Splatting. This method avoids the need for iterative updates, making it significantly more accurate and efficient compared to existing approaches. DGE also enables selective editing of specific parts of the scene, a capability not available in implicit radiance field representations. The paper includes a detailed description of the method, experimental results, and comparisons with other state-of-the-art methods, demonstrating the effectiveness and efficiency of DGE in 3D editing tasks.