Gaussian Splatting in Style

Gaussian Splatting in Style

6 Sep 2024 | Abhishek Saroha, Mariia Gladkova, Cecilia Curreli, Dominik Muhle, Tarun Yenamandra, and Daniel Cremers
This paper introduces a novel method for real-time 3D scene stylization called Gaussian Splatting in Style (GSS). Unlike previous approaches that require fitting a scene to each new style, GSS uses a neural network conditioned on a style image to generalize across various styles. The method employs 3D Gaussian splatting as the underlying 3D scene representation, which allows for fast training and rendering. The 3D Gaussians are processed using a multi-resolution hash grid and a tiny MLP to generate stylized views. The MLP is conditioned on different style codes to ensure generalization to different styles during test time. The explicit nature of 3D Gaussians provides inherent advantages over NeRF-based methods, including geometric consistency and fast training and rendering. This enables GSS to be useful for various practical applications such as augmented or virtual reality. The method achieves state-of-the-art performance with superior visual quality on various indoor and outdoor real-world data. The paper also compares GSS with existing methods and demonstrates its superiority in terms of consistency and rendering speed. GSS is able to generate stylized novel views at approximately 150 FPS, making it suitable for real-time applications. The method is evaluated on various datasets and shows better performance than existing baselines in both short-term and long-term consistency metrics. Additionally, the paper presents ablation studies and qualitative results showing the effectiveness of GSS in generating realistic and consistent stylized scenes.This paper introduces a novel method for real-time 3D scene stylization called Gaussian Splatting in Style (GSS). Unlike previous approaches that require fitting a scene to each new style, GSS uses a neural network conditioned on a style image to generalize across various styles. The method employs 3D Gaussian splatting as the underlying 3D scene representation, which allows for fast training and rendering. The 3D Gaussians are processed using a multi-resolution hash grid and a tiny MLP to generate stylized views. The MLP is conditioned on different style codes to ensure generalization to different styles during test time. The explicit nature of 3D Gaussians provides inherent advantages over NeRF-based methods, including geometric consistency and fast training and rendering. This enables GSS to be useful for various practical applications such as augmented or virtual reality. The method achieves state-of-the-art performance with superior visual quality on various indoor and outdoor real-world data. The paper also compares GSS with existing methods and demonstrates its superiority in terms of consistency and rendering speed. GSS is able to generate stylized novel views at approximately 150 FPS, making it suitable for real-time applications. The method is evaluated on various datasets and shows better performance than existing baselines in both short-term and long-term consistency metrics. Additionally, the paper presents ablation studies and qualitative results showing the effectiveness of GSS in generating realistic and consistent stylized scenes.
Reach us at info@study.space