SELF-REFINE: Iterative Refinement with Self-Feedback

SELF-REFINE: Iterative Refinement with Self-Feedback

25 May 2023 | Aman Madaan, Niket Tandon, Prakhar Gupta, Skyler Hallinan, Luyu Gao, Sarah Wiegreffe, Uri Alon, Nouha Dziri, Shrimai Prabhumoye, Yiming Yang, Shashank Gupta, Bodhisattwa Prasad Majumder, Katherine Hermann, Sean Welleck, Amir Yazdanbakhsh, Peter Clark
SELF-REFINE is an iterative self-refinement approach for improving initial outputs from large language models (LLMs) through feedback and refinement. The method uses a single LLM as the generator, refiner, and feedback provider, without requiring additional training or reinforcement learning. The process involves generating an initial output, then using the same LLM to provide feedback and refine the output iteratively until a stopping condition is met. SELF-REFINE has been evaluated across seven diverse tasks, including dialog response generation, code optimization, math reasoning, and sentiment reversal, using state-of-the-art LLMs such as GPT-3.5 and GPT-4. Results show that SELF-REFINE significantly improves task performance, with an average improvement of about 20% compared to conventional one-step generation. The approach is effective in enhancing the quality of outputs, even for models like GPT-4 that are already state-of-the-art. SELF-REFINE is simple, standalone, and can be applied to various tasks without additional training. The method relies on few-shot prompting and leverages the same LLM to generate feedback and refine outputs. The approach has been shown to be effective in improving outputs for a wide range of tasks, including code generation and natural language tasks. The results demonstrate that even when an LLM cannot generate an optimal output on its first try, it can provide useful feedback and improve its own output through iterative refinement. SELF-REFINE is a promising approach for improving the performance of LLMs in various tasks.SELF-REFINE is an iterative self-refinement approach for improving initial outputs from large language models (LLMs) through feedback and refinement. The method uses a single LLM as the generator, refiner, and feedback provider, without requiring additional training or reinforcement learning. The process involves generating an initial output, then using the same LLM to provide feedback and refine the output iteratively until a stopping condition is met. SELF-REFINE has been evaluated across seven diverse tasks, including dialog response generation, code optimization, math reasoning, and sentiment reversal, using state-of-the-art LLMs such as GPT-3.5 and GPT-4. Results show that SELF-REFINE significantly improves task performance, with an average improvement of about 20% compared to conventional one-step generation. The approach is effective in enhancing the quality of outputs, even for models like GPT-4 that are already state-of-the-art. SELF-REFINE is simple, standalone, and can be applied to various tasks without additional training. The method relies on few-shot prompting and leverages the same LLM to generate feedback and refine outputs. The approach has been shown to be effective in improving outputs for a wide range of tasks, including code generation and natural language tasks. The results demonstrate that even when an LLM cannot generate an optimal output on its first try, it can provide useful feedback and improve its own output through iterative refinement. SELF-REFINE is a promising approach for improving the performance of LLMs in various tasks.
Reach us at info@study.space