PALP: Prompt Aligned Personalization of Text-to-Image Models This paper introduces a novel approach for personalizing text-to-image models, focusing on prompt alignment. The method, called PALP, aims to generate images that are both personalized and aligned with a specific textual prompt. The challenge lies in balancing personalization with prompt alignment, as existing methods often compromise one for the other. PALP addresses this by fine-tuning a pre-trained model to learn a new subject while using score sampling to maintain alignment with the target prompt. The method is designed to handle complex prompts, including multiple subjects and styles, and can be applied in both single-shot and multi-shot settings. It also allows for the use of reference images, such as artworks, to inspire the generated images. The approach involves two main components: personalization, which teaches the model about the new subject, and prompt alignment, which ensures the model stays aligned with the target prompt. The method uses a score distillation sampling technique to guide the model's predictions towards the target prompt, improving text alignment while maintaining subject fidelity. Experiments show that PALP outperforms existing methods in terms of prompt alignment and personalization, particularly in complex scenarios. The method is efficient and can be applied to a wide range of subjects and prompts, making it a versatile solution for text-to-image generation. The results demonstrate that PALP achieves high-quality images that are both personalized and aligned with the target prompt, even when only a single reference image is available.PALP: Prompt Aligned Personalization of Text-to-Image Models This paper introduces a novel approach for personalizing text-to-image models, focusing on prompt alignment. The method, called PALP, aims to generate images that are both personalized and aligned with a specific textual prompt. The challenge lies in balancing personalization with prompt alignment, as existing methods often compromise one for the other. PALP addresses this by fine-tuning a pre-trained model to learn a new subject while using score sampling to maintain alignment with the target prompt. The method is designed to handle complex prompts, including multiple subjects and styles, and can be applied in both single-shot and multi-shot settings. It also allows for the use of reference images, such as artworks, to inspire the generated images. The approach involves two main components: personalization, which teaches the model about the new subject, and prompt alignment, which ensures the model stays aligned with the target prompt. The method uses a score distillation sampling technique to guide the model's predictions towards the target prompt, improving text alignment while maintaining subject fidelity. Experiments show that PALP outperforms existing methods in terms of prompt alignment and personalization, particularly in complex scenarios. The method is efficient and can be applied to a wide range of subjects and prompts, making it a versatile solution for text-to-image generation. The results demonstrate that PALP achieves high-quality images that are both personalized and aligned with the target prompt, even when only a single reference image is available.