10 Mar 2023 | Shunyu Yao, Jeffrey Zhao, Dian Yu, Nan Du, Izhak Shafran, Karthik Narasimhan, Yuan Cao
ReAct is a novel approach that combines reasoning and acting in large language models (LLMs) to solve diverse language reasoning and decision-making tasks. The method prompts LLMs to generate both verbal reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two. Reasoning traces help the model induce, track, and update action plans, while actions allow it to interface with and gather additional information from external sources. ReAct has been applied to a diverse set of language and decision-making tasks, demonstrating effectiveness over state-of-the-art baselines, improved human interpretability, and trustworthiness.
On question-answering tasks like HotPotQA and fact verification like Fever, ReAct overcomes issues of hallucination and error propagation in chain-of-thought reasoning by interacting with a simple Wikipedia API, generating human-like task-solving trajectories that are more interpretable than baselines without reasoning traces. On interactive decision-making benchmarks like ALFWorld and WebShop, ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples.
ReAct is designed to be intuitive, general, performant, and human-aligned. It allows for the integration of internal and external knowledge, and has been shown to outperform baselines in both knowledge-intensive reasoning tasks and decision-making tasks. ReAct also contributes to model interpretability, trustworthiness, and diagnostics across all domains, as humans can readily distinguish information from the model's internal knowledge versus external environments, and inspect reasoning traces to understand the decision basis of model actions.
The key contributions of ReAct include introducing a novel prompt-based paradigm to synergize reasoning and acting in language models for general task solving, performing extensive experiments across diverse benchmarks to showcase the advantage of ReAct in a few-shot learning setup over prior approaches that perform either reasoning or action generation in isolation, presenting systematic ablations and analysis to understand the importance of acting in reasoning tasks and reasoning in interactive tasks, and analyzing the limitations of ReAct under the prompting setup and performing initial fine-tuning experiments showing the potential of ReAct to improve with additional training data. Scaling up ReAct to train and operate on more tasks and combining it with complementary paradigms like reinforcement learning could further unlock the potential of large language models.ReAct is a novel approach that combines reasoning and acting in large language models (LLMs) to solve diverse language reasoning and decision-making tasks. The method prompts LLMs to generate both verbal reasoning traces and task-specific actions in an interleaved manner, allowing for greater synergy between the two. Reasoning traces help the model induce, track, and update action plans, while actions allow it to interface with and gather additional information from external sources. ReAct has been applied to a diverse set of language and decision-making tasks, demonstrating effectiveness over state-of-the-art baselines, improved human interpretability, and trustworthiness.
On question-answering tasks like HotPotQA and fact verification like Fever, ReAct overcomes issues of hallucination and error propagation in chain-of-thought reasoning by interacting with a simple Wikipedia API, generating human-like task-solving trajectories that are more interpretable than baselines without reasoning traces. On interactive decision-making benchmarks like ALFWorld and WebShop, ReAct outperforms imitation and reinforcement learning methods by an absolute success rate of 34% and 10% respectively, while being prompted with only one or two in-context examples.
ReAct is designed to be intuitive, general, performant, and human-aligned. It allows for the integration of internal and external knowledge, and has been shown to outperform baselines in both knowledge-intensive reasoning tasks and decision-making tasks. ReAct also contributes to model interpretability, trustworthiness, and diagnostics across all domains, as humans can readily distinguish information from the model's internal knowledge versus external environments, and inspect reasoning traces to understand the decision basis of model actions.
The key contributions of ReAct include introducing a novel prompt-based paradigm to synergize reasoning and acting in language models for general task solving, performing extensive experiments across diverse benchmarks to showcase the advantage of ReAct in a few-shot learning setup over prior approaches that perform either reasoning or action generation in isolation, presenting systematic ablations and analysis to understand the importance of acting in reasoning tasks and reasoning in interactive tasks, and analyzing the limitations of ReAct under the prompting setup and performing initial fine-tuning experiments showing the potential of ReAct to improve with additional training data. Scaling up ReAct to train and operate on more tasks and combining it with complementary paradigms like reinforcement learning could further unlock the potential of large language models.