2024 | Y Poirier-Ginter, A Gauthier, J Phillip, J.-F Lalonde, George Drettakis
The paper presents a novel method for creating relightable radiance fields from single-illumination multi-view datasets using multi-illumination synthesis. The authors leverage 2D image diffusion models to exploit priors and transform a single-illumination capture into a realistic multi-illumination dataset. This augmented data is then used to train a 3D Gaussian Splattering (3DGS) representation, enabling interactive relighting of complete scenes with low-frequency lighting control. The method addresses the underconstrained nature of radiance field relighting by fine-tuning a pre-trained 2D diffusion model on a multi-illumination dataset, allowing direct control over lighting direction. The augmented dataset is used to create a relightable radiance field that accounts for lighting inconsistencies across views. The effectiveness of the method is demonstrated on both synthetic and real-world scenes, showing realistic relighting results and improved performance compared to existing methods. The contributions include a 2D relighting neural network, a method to augment single-lighting multi-view captures, and an interactive relightable radiance field that corrects for lighting inconsistencies.The paper presents a novel method for creating relightable radiance fields from single-illumination multi-view datasets using multi-illumination synthesis. The authors leverage 2D image diffusion models to exploit priors and transform a single-illumination capture into a realistic multi-illumination dataset. This augmented data is then used to train a 3D Gaussian Splattering (3DGS) representation, enabling interactive relighting of complete scenes with low-frequency lighting control. The method addresses the underconstrained nature of radiance field relighting by fine-tuning a pre-trained 2D diffusion model on a multi-illumination dataset, allowing direct control over lighting direction. The augmented dataset is used to create a relightable radiance field that accounts for lighting inconsistencies across views. The effectiveness of the method is demonstrated on both synthetic and real-world scenes, showing realistic relighting results and improved performance compared to existing methods. The contributions include a 2D relighting neural network, a method to augment single-lighting multi-view captures, and an interactive relightable radiance field that corrects for lighting inconsistencies.