GaussianStego: A Generalizable Steganography Pipeline for Generative 3D Gaussians Splatting

GaussianStego: A Generalizable Steganography Pipeline for Generative 3D Gaussians Splatting

1 Jul 2024 | Chenxin Li¹; Hengyu Liu¹; Zhiwen Fan²; Wuyang Li¹; Yifan Liu¹; Panwang Pan³; Yixuan Yuan¹
GaussianStego is a novel steganographic method for embedding hidden information into the rendering of generated 3D assets, particularly in Gaussian Splatting. The method employs an optimization framework that enables the accurate extraction of hidden information from images rendered using Gaussian assets derived from large models, while maintaining their original visual quality. The approach integrates customizable, imperceptible, and recoverable information into the generative process without compromising visual quality or the generation pipeline. Unlike traditional image steganography that embeds hidden signals into specific source images, GaussianStego aims to recover the intended hidden signal from generated 3D Gaussians rendered from predetermined viewpoints for verification purposes. The method utilizes a vision foundation model to extract informative watermark embeddings, injecting them into the intermediate features of a 3D Gaussian generation baseline via cross-attention. For recovery, a U-Net-based decoder retrieves the concealed information from images rendered in specific verification viewpoints. Adaptive gradient harmonization is employed to constrain updates to weights where gradients of information hiding and rendering loss align, balancing rendering preservation and information hiding. Extensive experiments confirm the model's superiority in novel view synthesis rendering quality and high-fidelity transmission of hidden information. GaussianStego addresses the challenge of embedding customizable, imperceptible, and recoverable information within the renders produced by current 3D generative models, while ensuring minimal impact on the rendered content's quality. The method is evaluated across various deployment scenarios, demonstrating its effectiveness in embedding 2D visual contents and multimodal information such as text, QR codes, audio, and video. The framework is also tested for robustness against common perturbations like JPEG compression and Gaussian noise, showing consistent performance. The results highlight the potential of GaussianStego in practical 3D asset production environments, particularly in protecting intellectual property in 3D assets. The method's ability to generalize to unseen objects and its flexibility in handling multimodal information open up numerous potential applications for watermarking in generative 3D Gaussian contexts.GaussianStego is a novel steganographic method for embedding hidden information into the rendering of generated 3D assets, particularly in Gaussian Splatting. The method employs an optimization framework that enables the accurate extraction of hidden information from images rendered using Gaussian assets derived from large models, while maintaining their original visual quality. The approach integrates customizable, imperceptible, and recoverable information into the generative process without compromising visual quality or the generation pipeline. Unlike traditional image steganography that embeds hidden signals into specific source images, GaussianStego aims to recover the intended hidden signal from generated 3D Gaussians rendered from predetermined viewpoints for verification purposes. The method utilizes a vision foundation model to extract informative watermark embeddings, injecting them into the intermediate features of a 3D Gaussian generation baseline via cross-attention. For recovery, a U-Net-based decoder retrieves the concealed information from images rendered in specific verification viewpoints. Adaptive gradient harmonization is employed to constrain updates to weights where gradients of information hiding and rendering loss align, balancing rendering preservation and information hiding. Extensive experiments confirm the model's superiority in novel view synthesis rendering quality and high-fidelity transmission of hidden information. GaussianStego addresses the challenge of embedding customizable, imperceptible, and recoverable information within the renders produced by current 3D generative models, while ensuring minimal impact on the rendered content's quality. The method is evaluated across various deployment scenarios, demonstrating its effectiveness in embedding 2D visual contents and multimodal information such as text, QR codes, audio, and video. The framework is also tested for robustness against common perturbations like JPEG compression and Gaussian noise, showing consistent performance. The results highlight the potential of GaussianStego in practical 3D asset production environments, particularly in protecting intellectual property in 3D assets. The method's ability to generalize to unseen objects and its flexibility in handling multimodal information open up numerous potential applications for watermarking in generative 3D Gaussian contexts.
Reach us at info@study.space
Understanding GaussianStego%3A A Generalizable Stenography Pipeline for Generative 3D Gaussians Splatting