**TextGrad: Automatic “Differentiation” via Text** **Authors:** Mert Yükselgonull, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, James Zou **Institution:** Department of Computer Science, Stanford University; Department of Biomedical Data Science, Stanford University; Chan Zuckerberg Biohub **Abstract:** The development of compound AI systems, which integrate multiple large language models (LLMs) and other complex components, presents a significant challenge for optimization. Inspired by the success of automatic differentiation in neural networks, the authors introduce TextGrad, a framework that performs automatic "differentiation" via text. TextGrad uses textual feedback from LLMs to improve individual components of a compound AI system. The framework follows PyTorch's syntax and is flexible and easy to use. It can be applied to various tasks, such as question answering, molecule optimization, and radiotherapy treatment planning, without requiring users to tune components or provide specific prompts. **Key Contributions:** 1. **Automatic Differentiation via Text:** TextGrad allows for the backpropagation of textual feedback from LLMs to optimize individual components of a compound AI system. 2. **Flexibility and Generality:** The framework is built on PyTorch's abstractions, making it flexible and easy to use for a wide range of applications. 3. **State-of-the-Art Results:** TextGrad achieves state-of-the-art results in code optimization, PhD-level question answering, prompt optimization, and scientific applications such as molecule design and treatment planning. **Applications:** 1. **Code Optimization:** Improved performance in LeetCode Hard coding problems. 2. **Question Answering:** Enhanced zero-shot accuracy in Google-Proof Question Answering. 3. **Prompt Optimization:** Improved performance in reasoning tasks using LLMs. 4. **Molecule Optimization:** Designed new small molecules with desirable properties. 5. **Radiotherapy Treatment Planning:** Optimized treatment plans for prostate cancer patients. **Conclusion:** TextGrad lays the foundation for accelerating the development of next-generation AI systems by providing a powerful framework for automatic differentiation via text. The framework's flexibility and generality make it applicable to a wide range of applications, from scientific discovery to engineering tasks.**TextGrad: Automatic “Differentiation” via Text** **Authors:** Mert Yükselgonull, Federico Bianchi, Joseph Boen, Sheng Liu, Zhi Huang, Carlos Guestrin, James Zou **Institution:** Department of Computer Science, Stanford University; Department of Biomedical Data Science, Stanford University; Chan Zuckerberg Biohub **Abstract:** The development of compound AI systems, which integrate multiple large language models (LLMs) and other complex components, presents a significant challenge for optimization. Inspired by the success of automatic differentiation in neural networks, the authors introduce TextGrad, a framework that performs automatic "differentiation" via text. TextGrad uses textual feedback from LLMs to improve individual components of a compound AI system. The framework follows PyTorch's syntax and is flexible and easy to use. It can be applied to various tasks, such as question answering, molecule optimization, and radiotherapy treatment planning, without requiring users to tune components or provide specific prompts. **Key Contributions:** 1. **Automatic Differentiation via Text:** TextGrad allows for the backpropagation of textual feedback from LLMs to optimize individual components of a compound AI system. 2. **Flexibility and Generality:** The framework is built on PyTorch's abstractions, making it flexible and easy to use for a wide range of applications. 3. **State-of-the-Art Results:** TextGrad achieves state-of-the-art results in code optimization, PhD-level question answering, prompt optimization, and scientific applications such as molecule design and treatment planning. **Applications:** 1. **Code Optimization:** Improved performance in LeetCode Hard coding problems. 2. **Question Answering:** Enhanced zero-shot accuracy in Google-Proof Question Answering. 3. **Prompt Optimization:** Improved performance in reasoning tasks using LLMs. 4. **Molecule Optimization:** Designed new small molecules with desirable properties. 5. **Radiotherapy Treatment Planning:** Optimized treatment plans for prostate cancer patients. **Conclusion:** TextGrad lays the foundation for accelerating the development of next-generation AI systems by providing a powerful framework for automatic differentiation via text. The framework's flexibility and generality make it applicable to a wide range of applications, from scientific discovery to engineering tasks.