**StyleGaussian: Instant 3D Style Transfer with Gaussian Splatting** **Authors:** Kunhao Liu, Fangneng Zhan, Muyu Xu, Christian Theobalt, Ling Shao, Shijian Lu **Institution:** Nanyang Technological University, Max Planck Institute for Informatics, UCAS-Terminus AI Lab, UCAS **Abstract:** StyleGaussian is a novel 3D style transfer technique that enables instant style transfer of any image's style to a 3D scene at 10 frames per second (fps). It leverages 3D Gaussian Splatting (3DGS) to achieve real-time rendering and multi-view consistency without compromising style transfer quality. The process consists of three steps: embedding, transfer, and decoding. Initially, 2D VGG scene features are embedded into reconstructed 3D Gaussians. Next, the embedded features are transformed according to a reference style image. Finally, the transformed features are decoded into the stylized RGB. StyleGaussian introduces an efficient feature rendering strategy that first renders low-dimensional features and then maps them to high-dimensional features, significantly reducing memory consumption. It also employs a K-nearest-neighbor-based 3D CNN as the decoder, preserving strict multi-view consistency. Extensive experiments demonstrate superior stylization quality, real-time rendering, and multi-view consistency. **Contributions:** 1. Introduces StyleGaussian, a novel 3D style transfer pipeline for instant style transfer with real-time rendering and strict multi-view consistency. 2. Develops an efficient feature rendering strategy to handle high-dimensional VGG features. 3. Designs a KNN-based 3D CNN for decoding stylized features, maintaining multi-view consistency. **Keywords:** 3D Gaussian Splatting, 3D Style Transfer, 3D Editing **Related Work:** - **Radiance Fields:** Advances in radiance fields for 3D reconstruction. - **3D Appearance Editing:** Challenges and methods for editing 3D representations. - **Neural Style Transfer:** Techniques for neural style transfer in 2D and 3D. **Preliminary: 3D Gaussian Splatting** - Representation of 3D scenes using 3D Gaussians. - Rapid reconstruction and rendering capabilities. **StyleGaussian: Overview** - Embedding VGG features into 3D Gaussians. - Transforming features based on a style image. - Decoding transformed features back to RGB. **Feature Embedding:** - Embedding VGG features into 3D Gaussians. - Efficient rendering strategy for high-dimensional features. **Style Transfer:** - Using AdaIN for parameter-free style transfer. **RGB Decoding:** - KNN-based 3D CNN for decoding transformed features to RGB. **Experiments:** - Evaluations on real-world datasets. - Comparison with state-of-the-art methods.**StyleGaussian: Instant 3D Style Transfer with Gaussian Splatting** **Authors:** Kunhao Liu, Fangneng Zhan, Muyu Xu, Christian Theobalt, Ling Shao, Shijian Lu **Institution:** Nanyang Technological University, Max Planck Institute for Informatics, UCAS-Terminus AI Lab, UCAS **Abstract:** StyleGaussian is a novel 3D style transfer technique that enables instant style transfer of any image's style to a 3D scene at 10 frames per second (fps). It leverages 3D Gaussian Splatting (3DGS) to achieve real-time rendering and multi-view consistency without compromising style transfer quality. The process consists of three steps: embedding, transfer, and decoding. Initially, 2D VGG scene features are embedded into reconstructed 3D Gaussians. Next, the embedded features are transformed according to a reference style image. Finally, the transformed features are decoded into the stylized RGB. StyleGaussian introduces an efficient feature rendering strategy that first renders low-dimensional features and then maps them to high-dimensional features, significantly reducing memory consumption. It also employs a K-nearest-neighbor-based 3D CNN as the decoder, preserving strict multi-view consistency. Extensive experiments demonstrate superior stylization quality, real-time rendering, and multi-view consistency. **Contributions:** 1. Introduces StyleGaussian, a novel 3D style transfer pipeline for instant style transfer with real-time rendering and strict multi-view consistency. 2. Develops an efficient feature rendering strategy to handle high-dimensional VGG features. 3. Designs a KNN-based 3D CNN for decoding stylized features, maintaining multi-view consistency. **Keywords:** 3D Gaussian Splatting, 3D Style Transfer, 3D Editing **Related Work:** - **Radiance Fields:** Advances in radiance fields for 3D reconstruction. - **3D Appearance Editing:** Challenges and methods for editing 3D representations. - **Neural Style Transfer:** Techniques for neural style transfer in 2D and 3D. **Preliminary: 3D Gaussian Splatting** - Representation of 3D scenes using 3D Gaussians. - Rapid reconstruction and rendering capabilities. **StyleGaussian: Overview** - Embedding VGG features into 3D Gaussians. - Transforming features based on a style image. - Decoding transformed features back to RGB. **Feature Embedding:** - Embedding VGG features into 3D Gaussians. - Efficient rendering strategy for high-dimensional features. **Style Transfer:** - Using AdaIN for parameter-free style transfer. **RGB Decoding:** - KNN-based 3D CNN for decoding transformed features to RGB. **Experiments:** - Evaluations on real-world datasets. - Comparison with state-of-the-art methods.