ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields

ViCA-NeRF: View-Consistency-Aware 3D Editing of Neural Radiance Fields

1 Feb 2024 | Jiahua Dong Yu-Xiong Wang
ViCA-NeRF is a novel method for 3D editing using text instructions, achieving multi-view consistency and controllability. The key insight of ViCA-NeRF is to explicitly propagate editing information across different views through two sources of regularization: geometric regularization and learned regularization. Geometric regularization leverages depth information from NeRF to establish image correspondences between views, while learned regularization aligns latent codes in a 2D diffusion model between edited and unedited images. This method operates in two stages: the first stage blends edits from key views to create a preliminary 3D edit, and the second stage refines the scene's appearance using NeRF training. ViCA-NeRF demonstrates higher levels of consistency and detail compared to state-of-the-art methods, such as Instruct-NeRF2NeRF, and is significantly more efficient, running 3 times faster. The method is evaluated on various scenes and text prompts, showing its effectiveness in handling complex editing tasks.ViCA-NeRF is a novel method for 3D editing using text instructions, achieving multi-view consistency and controllability. The key insight of ViCA-NeRF is to explicitly propagate editing information across different views through two sources of regularization: geometric regularization and learned regularization. Geometric regularization leverages depth information from NeRF to establish image correspondences between views, while learned regularization aligns latent codes in a 2D diffusion model between edited and unedited images. This method operates in two stages: the first stage blends edits from key views to create a preliminary 3D edit, and the second stage refines the scene's appearance using NeRF training. ViCA-NeRF demonstrates higher levels of consistency and detail compared to state-of-the-art methods, such as Instruct-NeRF2NeRF, and is significantly more efficient, running 3 times faster. The method is evaluated on various scenes and text prompts, showing its effectiveness in handling complex editing tasks.
Reach us at info@study.space
Understanding ViCA-NeRF%3A View-Consistency-Aware 3D Editing of Neural Radiance Fields