This paper explores the theoretical limitations of Direct Preference Optimization (DPO), a method that derives reward signals directly from pairwise preference data to align Large Language Models (LLMs) with human preferences. Despite its effectiveness, DPO has been criticized for its sensitivity to the effectiveness of Supervised Fine-Tuning (SFT) and its hindrance to generating human-preferred responses. The authors use field theory to analyze the optimization process of DPO, focusing on the gradient vector field of the DPO loss function. They find that the DPO loss function decreases the probability of producing human-dispreferred data faster than it increases the probability of producing preferred data. This insight explains why DPO struggles to learn human-preferred responses and why it is sensitive to the effectiveness of SFT. The paper provides a theoretical foundation for understanding and improving DPO, highlighting the need for more comprehensive theoretical analysis to address its limitations.This paper explores the theoretical limitations of Direct Preference Optimization (DPO), a method that derives reward signals directly from pairwise preference data to align Large Language Models (LLMs) with human preferences. Despite its effectiveness, DPO has been criticized for its sensitivity to the effectiveness of Supervised Fine-Tuning (SFT) and its hindrance to generating human-preferred responses. The authors use field theory to analyze the optimization process of DPO, focusing on the gradient vector field of the DPO loss function. They find that the DPO loss function decreases the probability of producing human-dispreferred data faster than it increases the probability of producing preferred data. This insight explains why DPO struggles to learn human-preferred responses and why it is sensitive to the effectiveness of SFT. The paper provides a theoretical foundation for understanding and improving DPO, highlighting the need for more comprehensive theoretical analysis to address its limitations.