This paper explores the potential of using large language models (LLMs) as interactive optimizers for solving maximization problems in text spaces using natural language and numerical feedback. The authors classify natural language feedback into directional and non-directional types, where directional feedback generalizes first-order feedback to the text space. They find that LLMs are particularly effective when provided with directional feedback. Based on this insight, they design a new LLM-based optimizer that synthesizes directional feedback from historical optimization traces to achieve reliable improvement over iterations. Empirically, they show that their LLM-based optimizer is more stable and efficient in solving optimization problems, from maximizing mathematical functions to optimizing prompts for writing poems, compared to existing techniques. The paper introduces a framework for prompt optimization in LLM-based agents, where the agent's behavior is modulated by prompts. The optimization process involves an iterative solver that improves over time by incorporating feedback. The authors propose an algorithm for sequential prompt optimization, which uses historical data to update the prompt and improve the agent's performance. They also explore the role of feedback in LLM-based text optimization, showing that directional feedback is crucial for effective optimization. In numerical optimization experiments, the authors test the ability of LLMs to implicitly perform Newton's method using historical data. They find that LLMs can improve their search direction when provided with directional feedback, and that synthetic feedback can be used to enhance the optimization process. In poem generation experiments, they validate their optimization setup on a practical domain, showing that their algorithm can reliably select prompts that improve policy performance for each task. The paper concludes that LLMs can successfully optimize a wide range of entities, from mathematical functions to prompts for textual tasks, when provided with directional feedback. They emphasize that this is an early work on general LLM-based optimizers, and that LLMs' potential in this role is still to be realized with new methods for directional feedback generation.This paper explores the potential of using large language models (LLMs) as interactive optimizers for solving maximization problems in text spaces using natural language and numerical feedback. The authors classify natural language feedback into directional and non-directional types, where directional feedback generalizes first-order feedback to the text space. They find that LLMs are particularly effective when provided with directional feedback. Based on this insight, they design a new LLM-based optimizer that synthesizes directional feedback from historical optimization traces to achieve reliable improvement over iterations. Empirically, they show that their LLM-based optimizer is more stable and efficient in solving optimization problems, from maximizing mathematical functions to optimizing prompts for writing poems, compared to existing techniques. The paper introduces a framework for prompt optimization in LLM-based agents, where the agent's behavior is modulated by prompts. The optimization process involves an iterative solver that improves over time by incorporating feedback. The authors propose an algorithm for sequential prompt optimization, which uses historical data to update the prompt and improve the agent's performance. They also explore the role of feedback in LLM-based text optimization, showing that directional feedback is crucial for effective optimization. In numerical optimization experiments, the authors test the ability of LLMs to implicitly perform Newton's method using historical data. They find that LLMs can improve their search direction when provided with directional feedback, and that synthetic feedback can be used to enhance the optimization process. In poem generation experiments, they validate their optimization setup on a practical domain, showing that their algorithm can reliably select prompts that improve policy performance for each task. The paper concludes that LLMs can successfully optimize a wide range of entities, from mathematical functions to prompts for textual tasks, when provided with directional feedback. They emphasize that this is an early work on general LLM-based optimizers, and that LLMs' potential in this role is still to be realized with new methods for directional feedback generation.