17 May 2024 | Pengzhi Li, Chengshuai Tang, Qinxuan Huang, Zhiheng Li
ART3D is a novel framework for generating 3D artistic scenes based on text or reference images. It combines diffusion models and 3D Gaussian splatting techniques to produce high-quality 3D artistic scenes. The framework addresses the challenge of generating 3D scenes from text descriptions by leveraging depth information and an initial artistic image to generate a point cloud map, bridging the gap between artistic and realistic images. A depth consistency module is introduced to enhance 3D scene consistency. The 3D scene serves as initial points for optimizing Gaussian splats. Experimental results show that ART3D outperforms existing methods in both content and structural consistency metrics. The method is extended to various applications, including text-driven fusion of different artistic styles. ART3D contributes to the intersection of art and technology by providing an innovative solution for generating high-quality 3D artistic scenes. The key components of ART3D include an image semantic transfer algorithm, a point cloud map, and a depth consistency module. The image semantic transfer algorithm aligns the semantic information of artistic and realistic images. The point cloud map is used to generate 3D scenes. The depth consistency module ensures consistency between multiple views. The method is evaluated using quantitative metrics and user studies, demonstrating its effectiveness in generating structurally consistent and diverse 3D artistic scenes. The results show that ART3D achieves high-quality 3D artistic scenes with superior performance compared to existing methods.ART3D is a novel framework for generating 3D artistic scenes based on text or reference images. It combines diffusion models and 3D Gaussian splatting techniques to produce high-quality 3D artistic scenes. The framework addresses the challenge of generating 3D scenes from text descriptions by leveraging depth information and an initial artistic image to generate a point cloud map, bridging the gap between artistic and realistic images. A depth consistency module is introduced to enhance 3D scene consistency. The 3D scene serves as initial points for optimizing Gaussian splats. Experimental results show that ART3D outperforms existing methods in both content and structural consistency metrics. The method is extended to various applications, including text-driven fusion of different artistic styles. ART3D contributes to the intersection of art and technology by providing an innovative solution for generating high-quality 3D artistic scenes. The key components of ART3D include an image semantic transfer algorithm, a point cloud map, and a depth consistency module. The image semantic transfer algorithm aligns the semantic information of artistic and realistic images. The point cloud map is used to generate 3D scenes. The depth consistency module ensures consistency between multiple views. The method is evaluated using quantitative metrics and user studies, demonstrating its effectiveness in generating structurally consistent and diverse 3D artistic scenes. The results show that ART3D achieves high-quality 3D artistic scenes with superior performance compared to existing methods.