Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning

Rethink Arbitrary Style Transfer with Transformer and Contrastive Learning

2024 | Zhanjie Zhang, Jiakai Sun, Guangyuan Li, Lei Zhao, Quanwei Zhang, Zehua Lan, Haolin Yin, Wei Xing, Huaizhong Lin, Zhiwen Zuo
This paper addresses the challenge of arbitrary style transfer, a technique that generates new images by incorporating the style of one image into another while retaining its content. The authors propose an innovative method to improve the quality of stylized images, introducing Style Consistency Instance Normalization (SCIN) and Instance-based Contrastive Learning (ICL). SCIN refines the alignment between content and style features, while ICL enhances the quality of stylized images by learning the relationships among various styles. Additionally, the authors introduce a Perception Encoder (PE) to better capture style features, as VGG networks are better suited for extracting classification features rather than style features. Extensive experiments demonstrate that the proposed method generates high-quality stylized images with fewer artifacts compared to existing state-of-the-art methods. The main contributions of the paper include the introduction of SCIN, ICL, and PE, which collectively improve the quality and realism of stylized images.This paper addresses the challenge of arbitrary style transfer, a technique that generates new images by incorporating the style of one image into another while retaining its content. The authors propose an innovative method to improve the quality of stylized images, introducing Style Consistency Instance Normalization (SCIN) and Instance-based Contrastive Learning (ICL). SCIN refines the alignment between content and style features, while ICL enhances the quality of stylized images by learning the relationships among various styles. Additionally, the authors introduce a Perception Encoder (PE) to better capture style features, as VGG networks are better suited for extracting classification features rather than style features. Extensive experiments demonstrate that the proposed method generates high-quality stylized images with fewer artifacts compared to existing state-of-the-art methods. The main contributions of the paper include the introduction of SCIN, ICL, and PE, which collectively improve the quality and realism of stylized images.
Reach us at info@study.space