Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion

Sat2Scene: 3D Urban Scene Generation from Satellite Images with Diffusion

1 Apr 2024 | Zuoyue Li, Zhenqiang Li, Zhaopeng Cui, Marc Pollefeys, Martin R. Oswald
Sat2Scene is a novel architecture for generating 3D urban scenes from satellite images, addressing the challenges of significant view changes and scene scale. The method combines diffusion models with 3D sparse representations and neural rendering techniques. Specifically, it generates texture colors at the point level using a 3D diffusion model, which is then transformed into a scene representation for rendering arbitrary views. This approach ensures both single-frame quality and inter-frame consistency. Experiments on two city-scale datasets, HoliCity and OmniCity, demonstrate the model's proficiency in generating photorealistic street-view image sequences and cross-view urban scenes. The contributions of the paper include a novel diffusion-based framework, a diffusion model with sparse representations, and superior performance in generating photorealistic street-view image sequences and cross-view urban scenes. The method outperforms state-of-the-art baselines in terms of overall video quality and inter-frame consistency, showcasing a balance between photorealism and consistency.Sat2Scene is a novel architecture for generating 3D urban scenes from satellite images, addressing the challenges of significant view changes and scene scale. The method combines diffusion models with 3D sparse representations and neural rendering techniques. Specifically, it generates texture colors at the point level using a 3D diffusion model, which is then transformed into a scene representation for rendering arbitrary views. This approach ensures both single-frame quality and inter-frame consistency. Experiments on two city-scale datasets, HoliCity and OmniCity, demonstrate the model's proficiency in generating photorealistic street-view image sequences and cross-view urban scenes. The contributions of the paper include a novel diffusion-based framework, a diffusion model with sparse representations, and superior performance in generating photorealistic street-view image sequences and cross-view urban scenes. The method outperforms state-of-the-art baselines in terms of overall video quality and inter-frame consistency, showcasing a balance between photorealism and consistency.
Reach us at info@study.space
Understanding Sat2Scene%3A 3D Urban Scene Generation from Satellite Images with Diffusion