This paper provides a theoretical analysis of the limitations of Direct Preference Optimization (DPO), focusing on its sensitivity to the effectiveness of Supervised Fine-Tuning (SFT) and its hindrance to the learning capacity of Large Language Models (LLMs) in generating human-preferred responses. DPO, which directly derives reward signals from pairwise preference data, has shown promise in aligning LLMs with human preferences. However, it has been criticized for its sensitivity to SFT effectiveness and for its tendency to prioritize reducing the probability of generating human-dispreferred responses over increasing the probability of generating human-preferred responses. Using field theory, the paper analyzes the gradient vector field of the DPO loss function, revealing that the loss function decreases the probability of generating human-dispreferred data at a faster rate than it increases the probability of generating preferred data. This suggests that DPO may be more effective at avoiding dispreferred responses than at promoting preferred ones, which can hinder the learning capacity of LLMs to generate human-preferred responses. The paper also highlights that the effectiveness of DPO is closely tied to the alignment capability of LLMs after SFT. If SFT is not effective, DPO may not perform well, as the initial conditions of the LLMs in the optimization plane can significantly impact the results. The analysis shows that the gradient vector field of DPO varies in magnitude and direction, which can lead to different optimization outcomes depending on the initial conditions. In conclusion, the paper provides a theoretical framework for understanding the limitations of DPO, emphasizing the need for further research to improve its effectiveness in aligning LLMs with human preferences. The findings suggest that DPO's limitations are rooted in its optimization process, which may be sensitive to the initial conditions and the effectiveness of SFT. This understanding can guide future improvements to DPO and related methods.This paper provides a theoretical analysis of the limitations of Direct Preference Optimization (DPO), focusing on its sensitivity to the effectiveness of Supervised Fine-Tuning (SFT) and its hindrance to the learning capacity of Large Language Models (LLMs) in generating human-preferred responses. DPO, which directly derives reward signals from pairwise preference data, has shown promise in aligning LLMs with human preferences. However, it has been criticized for its sensitivity to SFT effectiveness and for its tendency to prioritize reducing the probability of generating human-dispreferred responses over increasing the probability of generating human-preferred responses. Using field theory, the paper analyzes the gradient vector field of the DPO loss function, revealing that the loss function decreases the probability of generating human-dispreferred data at a faster rate than it increases the probability of generating preferred data. This suggests that DPO may be more effective at avoiding dispreferred responses than at promoting preferred ones, which can hinder the learning capacity of LLMs to generate human-preferred responses. The paper also highlights that the effectiveness of DPO is closely tied to the alignment capability of LLMs after SFT. If SFT is not effective, DPO may not perform well, as the initial conditions of the LLMs in the optimization plane can significantly impact the results. The analysis shows that the gradient vector field of DPO varies in magnitude and direction, which can lead to different optimization outcomes depending on the initial conditions. In conclusion, the paper provides a theoretical framework for understanding the limitations of DPO, emphasizing the need for further research to improve its effectiveness in aligning LLMs with human preferences. The findings suggest that DPO's limitations are rooted in its optimization process, which may be sensitive to the initial conditions and the effectiveness of SFT. This understanding can guide future improvements to DPO and related methods.