ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields

ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields

2024 | Jiahua Dong, Yu-Xiong Wang
ViCA-NeRF is a novel method for 3D editing of Neural Radiance Fields (NeRF) with text instructions, achieving multi-view consistency. Unlike previous methods, ViCA-NeRF introduces two regularization strategies: geometric and learned regularization. Geometric regularization uses depth information from NeRF to align image correspondences across views, while learned regularization aligns latent codes in a 2D diffusion model between edited and unedited images. These strategies enable efficient and controllable 3D editing, with ViCA-NeRF being three times faster than state-of-the-art Instruct-NeRF2NeRF. The method operates in two stages: first, edits from different views are blended to create a preliminary 3D edit, followed by NeRF training to refine the scene. Experimental results show that ViCA-NeRF provides more consistent and detailed edits compared to existing methods. The method is evaluated on various scenes and text prompts, demonstrating its effectiveness in handling complex editing tasks. ViCA-NeRF also supports local editing, allowing for targeted modifications without affecting the background. The method's efficiency and consistency make it a significant advancement in 3D editing with text instructions.ViCA-NeRF is a novel method for 3D editing of Neural Radiance Fields (NeRF) with text instructions, achieving multi-view consistency. Unlike previous methods, ViCA-NeRF introduces two regularization strategies: geometric and learned regularization. Geometric regularization uses depth information from NeRF to align image correspondences across views, while learned regularization aligns latent codes in a 2D diffusion model between edited and unedited images. These strategies enable efficient and controllable 3D editing, with ViCA-NeRF being three times faster than state-of-the-art Instruct-NeRF2NeRF. The method operates in two stages: first, edits from different views are blended to create a preliminary 3D edit, followed by NeRF training to refine the scene. Experimental results show that ViCA-NeRF provides more consistent and detailed edits compared to existing methods. The method is evaluated on various scenes and text prompts, demonstrating its effectiveness in handling complex editing tasks. ViCA-NeRF also supports local editing, allowing for targeted modifications without affecting the background. The method's efficiency and consistency make it a significant advancement in 3D editing with text instructions.
Reach us at info@study.space