Are Large Language Models Good Prompt Optimizers? This paper investigates the effectiveness of large language models (LLMs) as prompt optimizers. While LLM-based automatic prompt optimization has shown promise, the underlying mechanisms remain unclear. The study reveals that LLM optimizers often fail to identify true causes of errors during reflection, instead relying on prior knowledge. Additionally, even when reflection is semantically valid, LLMs often fail to generate appropriate prompts for target models in a single refinement step, partly due to unpredictable behaviors of the target models. The paper introduces a new "Automatic Behavior Optimization" paradigm that directly optimizes the behavior of target models in a more controllable manner. The results show that this approach is effective in improving the performance of less powerful target models. The study highlights the limitations of current LLM-based prompt optimization methods and advocates for new paradigms that can better address these issues. The findings suggest that while LLMs can perform some form of reflection, they often lack the ability to generate effective prompts, and the effectiveness of prompt optimization depends on the alignment between the LLM's behavior and the target model's instruction-following capabilities. The paper concludes that further research is needed to develop more effective prompt optimization methods.Are Large Language Models Good Prompt Optimizers? This paper investigates the effectiveness of large language models (LLMs) as prompt optimizers. While LLM-based automatic prompt optimization has shown promise, the underlying mechanisms remain unclear. The study reveals that LLM optimizers often fail to identify true causes of errors during reflection, instead relying on prior knowledge. Additionally, even when reflection is semantically valid, LLMs often fail to generate appropriate prompts for target models in a single refinement step, partly due to unpredictable behaviors of the target models. The paper introduces a new "Automatic Behavior Optimization" paradigm that directly optimizes the behavior of target models in a more controllable manner. The results show that this approach is effective in improving the performance of less powerful target models. The study highlights the limitations of current LLM-based prompt optimization methods and advocates for new paradigms that can better address these issues. The findings suggest that while LLMs can perform some form of reflection, they often lack the ability to generate effective prompts, and the effectiveness of prompt optimization depends on the alignment between the LLM's behavior and the target model's instruction-following capabilities. The paper concludes that further research is needed to develop more effective prompt optimization methods.