Semantic Image Synthesis with Spatially-Adaptive Normalization

Semantic Image Synthesis with Spatially-Adaptive Normalization

5 Nov 2019 | Taesung Park, Ming-Yu Liu, Ting-Chun Wang, Jun-Yan Zhu
The paper introduces a novel method called Spatially-Adaptive Normalization (SPADE) for synthesizing photorealistic images from semantic layouts. Traditional methods often struggle with preserving semantic information during normalization, which can lead to loss of detail and realism. SPADE addresses this issue by using a learned transformation to modulate the activations in normalization layers, ensuring that semantic information is maintained throughout the network. The proposed method is evaluated on several challenging datasets, including COCO-Stuff, ADE20K, and Cityscapes, demonstrating superior performance in terms of visual fidelity and alignment with input layouts. Additionally, the model supports multi-modal and style-guided image synthesis, allowing users to control both semantic and style aspects of the synthesized images. The effectiveness of SPADE is demonstrated through extensive experiments and ablation studies, showing that it outperforms existing methods in various metrics such as mIoU, FID scores, and user preference studies.The paper introduces a novel method called Spatially-Adaptive Normalization (SPADE) for synthesizing photorealistic images from semantic layouts. Traditional methods often struggle with preserving semantic information during normalization, which can lead to loss of detail and realism. SPADE addresses this issue by using a learned transformation to modulate the activations in normalization layers, ensuring that semantic information is maintained throughout the network. The proposed method is evaluated on several challenging datasets, including COCO-Stuff, ADE20K, and Cityscapes, demonstrating superior performance in terms of visual fidelity and alignment with input layouts. Additionally, the model supports multi-modal and style-guided image synthesis, allowing users to control both semantic and style aspects of the synthesized images. The effectiveness of SPADE is demonstrated through extensive experiments and ablation studies, showing that it outperforms existing methods in various metrics such as mIoU, FID scores, and user preference studies.
Reach us at info@study.space
[slides] Semantic Image Synthesis With Spatially-Adaptive Normalization | StudySpace