DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading

DeferredGS: Decoupled and Editable Gaussian Splatting with Deferred Shading

March 2024 | Tong Wu, Jia-Mu Sun, Yu-Kun Lai, Yuewen Ma, Leif Kobbelt and Lin Gao
DeferredGS is a method for decoupling and editing Gaussian splatting using deferred shading. It addresses the limitations of traditional Gaussian splatting, which combines texture and lighting into a single spherical harmonic function, limiting independent editing. DeferredGS introduces a learnable environment map to model illumination and defines additional attributes like texture parameters and normal direction on Gaussians, with the normal derived from a signed distance function. It applies deferred shading to achieve more realistic relighting effects. Qualitative and quantitative experiments show that DeferredGS outperforms previous methods in novel view synthesis and editing tasks. The method enables geometry, texture, and lighting editing by decoupling these components. DeferredGS uses deferred shading to separate geometry and texture attributes into buffers, allowing for accurate normal estimation and reducing blending artifacts during relighting. It also supports geometry and texture editing by deforming Gaussians based on a mesh proxy and modifying texture attributes to achieve desired changes. The method achieves real-time rendering at 30FPS on a single 3090 GPU. DeferredGS is the first to apply deferred shading to Gaussian splatting, improving relighting results and reducing artifacts. It outperforms other methods in decomposition, relighting, and editing tasks, with quantitative results showing higher PSNR, SSIM, and LPIPS scores. The method also handles scenes with shadows and noisy textures, though it may produce incorrect decoupled results in such cases. Future work could involve capturing scenes under multiple lighting conditions and incorporating visibility into the rendering process.DeferredGS is a method for decoupling and editing Gaussian splatting using deferred shading. It addresses the limitations of traditional Gaussian splatting, which combines texture and lighting into a single spherical harmonic function, limiting independent editing. DeferredGS introduces a learnable environment map to model illumination and defines additional attributes like texture parameters and normal direction on Gaussians, with the normal derived from a signed distance function. It applies deferred shading to achieve more realistic relighting effects. Qualitative and quantitative experiments show that DeferredGS outperforms previous methods in novel view synthesis and editing tasks. The method enables geometry, texture, and lighting editing by decoupling these components. DeferredGS uses deferred shading to separate geometry and texture attributes into buffers, allowing for accurate normal estimation and reducing blending artifacts during relighting. It also supports geometry and texture editing by deforming Gaussians based on a mesh proxy and modifying texture attributes to achieve desired changes. The method achieves real-time rendering at 30FPS on a single 3090 GPU. DeferredGS is the first to apply deferred shading to Gaussian splatting, improving relighting results and reducing artifacts. It outperforms other methods in decomposition, relighting, and editing tasks, with quantitative results showing higher PSNR, SSIM, and LPIPS scores. The method also handles scenes with shadows and noisy textures, though it may produce incorrect decoupled results in such cases. Future work could involve capturing scenes under multiple lighting conditions and incorporating visibility into the rendering process.
Reach us at info@study.space
Understanding DeferredGS%3A Decoupled and Editable Gaussian Splatting with Deferred Shading