**KNOWAGENT: Knowledge-Augmented Planning for LLM-Based Agents** **Authors:** Yuqi Zhu, Shuohei Qiao, Yixin Ou, Shumin Deng, Ningyu Zhang, Shiwei Lyu, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen **Institution:** Zhejiang University ZJU-Ant Group Joint Research Center for Knowledge Graphs; Ant Group National University of Singapore **Abstract:** Large Language Models (LLMs) excel in complex reasoning tasks but struggle with sophisticated challenges, especially when generating executable actions. This limitation stems from the lack of built-in action knowledge, leading to *planning hallucination*. To address this, KNOWAGENT incorporates explicit action knowledge to enhance planning capabilities. KNOWAGENT uses an *action knowledge base* and a *knowledgeable self-learning* strategy to guide action paths, improving trajectory synthesis and planning performance. Experiments on HotpotQA and ALFWorld show that KNOWAGENT achieves comparable or superior performance to existing baselines, effectively mitigating planning hallucinations. **Introduction:** Language agents, built around LLMs, enhance task planning through various strategies. However, they often generate plans that violate established knowledge rules or commonsense, a phenomenon called *planning hallucination*. KNOWAGENT addresses this issue by leveraging external action knowledge to refine synthetic trajectories. The method involves creating an extensive action knowledge base, converting it into text, and using a self-learning phase to iteratively improve trajectory quality. **KNOWAGENT:** - **Action Knowledge Base:** Amasses task-specific action knowledge. - **Action Knowledge to Text:** Converts action knowledge into textual descriptions. - **Planning Path Generation:** Uses prompts and action knowledge to guide trajectory creation. - **Knowledgeable Self-Learning:** Iteratively optimizes trajectories based on feedback. **Experiments:** - **Settings:** Evaluations on HotpotQA and ALFWorld using Llama-2, Vicuna, and Mistral models. - **Results:** KNOWAGENT outperforms prompt-based methods and fine-tuning methods, reducing invalid and misordered actions. **Analysis:** - **Ablation Study:** Action knowledge significantly improves trajectory quality. - **Iterative Training:** Enhances model proficiency with more iterations. - **Error Analysis:** LIMITED INTEGRATION OF LONG-TEXT DATA AND REASONING CAPABILITIES. **Conclusion:** KNOWAGENT effectively mitigates planning hallucinations by incorporating external action knowledge, enhancing planning capabilities in complex scenarios. Future work should explore broader applicability, multi-agent systems, and automated design of action knowledge bases.**KNOWAGENT: Knowledge-Augmented Planning for LLM-Based Agents** **Authors:** Yuqi Zhu, Shuohei Qiao, Yixin Ou, Shumin Deng, Ningyu Zhang, Shiwei Lyu, Yue Shen, Lei Liang, Jinjie Gu, Huajun Chen **Institution:** Zhejiang University ZJU-Ant Group Joint Research Center for Knowledge Graphs; Ant Group National University of Singapore **Abstract:** Large Language Models (LLMs) excel in complex reasoning tasks but struggle with sophisticated challenges, especially when generating executable actions. This limitation stems from the lack of built-in action knowledge, leading to *planning hallucination*. To address this, KNOWAGENT incorporates explicit action knowledge to enhance planning capabilities. KNOWAGENT uses an *action knowledge base* and a *knowledgeable self-learning* strategy to guide action paths, improving trajectory synthesis and planning performance. Experiments on HotpotQA and ALFWorld show that KNOWAGENT achieves comparable or superior performance to existing baselines, effectively mitigating planning hallucinations. **Introduction:** Language agents, built around LLMs, enhance task planning through various strategies. However, they often generate plans that violate established knowledge rules or commonsense, a phenomenon called *planning hallucination*. KNOWAGENT addresses this issue by leveraging external action knowledge to refine synthetic trajectories. The method involves creating an extensive action knowledge base, converting it into text, and using a self-learning phase to iteratively improve trajectory quality. **KNOWAGENT:** - **Action Knowledge Base:** Amasses task-specific action knowledge. - **Action Knowledge to Text:** Converts action knowledge into textual descriptions. - **Planning Path Generation:** Uses prompts and action knowledge to guide trajectory creation. - **Knowledgeable Self-Learning:** Iteratively optimizes trajectories based on feedback. **Experiments:** - **Settings:** Evaluations on HotpotQA and ALFWorld using Llama-2, Vicuna, and Mistral models. - **Results:** KNOWAGENT outperforms prompt-based methods and fine-tuning methods, reducing invalid and misordered actions. **Analysis:** - **Ablation Study:** Action knowledge significantly improves trajectory quality. - **Iterative Training:** Enhances model proficiency with more iterations. - **Error Analysis:** LIMITED INTEGRATION OF LONG-TEXT DATA AND REASONING CAPABILITIES. **Conclusion:** KNOWAGENT effectively mitigates planning hallucinations by incorporating external action knowledge, enhancing planning capabilities in complex scenarios. Future work should explore broader applicability, multi-agent systems, and automated design of action knowledge bases.