1 Apr 2024 | Gowthami Somepalli, Anubhav Gupta, Kamal Gupta, Shramay Palta, Micah Goldblum, Jonas Geiping, Abhinav Shrivastava, Tom Goldstein
This paper introduces a framework for measuring style similarity in diffusion models, focusing on extracting style descriptors from images. The authors propose a new dataset, LAION-Styles, which associates images with their artistic styles. They also introduce a method to extract style descriptors that can be used to attribute the style of a generated image to the training data of a text-to-image model. The framework includes a contrastive learning scheme for style descriptor extraction and is evaluated on various style retrieval tasks. The authors also analyze the style replication in the Stable Diffusion model and find that some artists' styles are not present in the model's training data. The results show that their model, CSD, outperforms other models in style matching tasks. The study highlights the importance of style attribution in understanding the relationship between generated images and their training data. The authors also discuss the challenges of style detection and retrieval, and the limitations of current methods. The paper concludes that style similarity can be used to assess the extent to which a model emulates an artist's style, and that the proposed method provides a more accurate and effective way to measure style similarity in diffusion models.This paper introduces a framework for measuring style similarity in diffusion models, focusing on extracting style descriptors from images. The authors propose a new dataset, LAION-Styles, which associates images with their artistic styles. They also introduce a method to extract style descriptors that can be used to attribute the style of a generated image to the training data of a text-to-image model. The framework includes a contrastive learning scheme for style descriptor extraction and is evaluated on various style retrieval tasks. The authors also analyze the style replication in the Stable Diffusion model and find that some artists' styles are not present in the model's training data. The results show that their model, CSD, outperforms other models in style matching tasks. The study highlights the importance of style attribution in understanding the relationship between generated images and their training data. The authors also discuss the challenges of style detection and retrieval, and the limitations of current methods. The paper concludes that style similarity can be used to assess the extent to which a model emulates an artist's style, and that the proposed method provides a more accurate and effective way to measure style similarity in diffusion models.