This paper addresses the issue of reference mismatch in aligning text-to-image diffusion models, particularly Stable Diffusion XL (SDXL), which occurs when the reference model and preference data have distinct features. The authors propose a novel method called Margin-Aware Preference Optimization (MaPO) that does not rely on a reference model, thereby avoiding the limitations of existing methods that use divergence regularization. MaPO jointly maximizes the likelihood margin between preferred and dispreferred image sets while learning general stylistic features and preferences. The paper introduces two new datasets, *Pick-Style* and *Pick-Safety*, to simulate different scenarios of reference mismatch. Empirical results show that MaPO significantly improves alignment on these datasets compared to existing methods, achieving higher win rates and better general preference alignment. Additionally, MaPO demonstrates superior computational efficiency, reducing training time by 14.5%. The authors conclude that MaPO is a flexible and memory-friendly method for aligning text-to-image diffusion models, applicable to various domain-specific preference data.This paper addresses the issue of reference mismatch in aligning text-to-image diffusion models, particularly Stable Diffusion XL (SDXL), which occurs when the reference model and preference data have distinct features. The authors propose a novel method called Margin-Aware Preference Optimization (MaPO) that does not rely on a reference model, thereby avoiding the limitations of existing methods that use divergence regularization. MaPO jointly maximizes the likelihood margin between preferred and dispreferred image sets while learning general stylistic features and preferences. The paper introduces two new datasets, *Pick-Style* and *Pick-Safety*, to simulate different scenarios of reference mismatch. Empirical results show that MaPO significantly improves alignment on these datasets compared to existing methods, achieving higher win rates and better general preference alignment. Additionally, MaPO demonstrates superior computational efficiency, reducing training time by 14.5%. The authors conclude that MaPO is a flexible and memory-friendly method for aligning text-to-image diffusion models, applicable to various domain-specific preference data.