Agent Planning with World Knowledge Model

Agent Planning with World Knowledge Model

23 May 2024 | Shuofei Qiao; Runnan Fang; Ningyu Zhang; Yuqi Zhu; Xiang Chen; Shumin Deng; Yong Jiang; Pengjun Xie; Fei Huang; Huajun Chen
This paper introduces a parametric World Knowledge Model (WKM) to enhance agent planning by incorporating global and local knowledge. The WKM is designed to provide prior task knowledge for global planning and dynamic state knowledge for local planning, addressing the limitations of large language models (LLMs) in understanding the physical world. The WKM is trained using knowledge synthesized from expert and sampled trajectories, enabling the agent to self-synthesize task knowledge and summarize state knowledge for each planning step. The agent model is then retrained to adapt to the task knowledge, with both the agent and knowledge models trained using LoRA. During planning, the WKM provides global prior knowledge and maintains dynamic state knowledge, guiding the agent to avoid blind trial-and-error and hallucinatory actions. The agent uses the WKM to retrieve relevant state knowledge from a pre-built knowledge base, combining it with the agent's own predictions to make informed decisions. Experimental results on three real-world simulated datasets (ALFWorld, WebShop, and ScienceWorld) with three state-of-the-art open-source LLMs (Mistral-7B, Gemma-7B, and Llama-3-8B) show that the proposed method outperforms various strong baselines. The WKM effectively reduces blind trial-and-error and hallucinatory actions, generalizes better to unseen tasks, and demonstrates the feasibility of weak-guide-strong planning. Additionally, the WKM shows potential for unified training across multiple tasks and highlights the importance of implicit knowledge constraints in agent planning. The results indicate that the WKM significantly improves agent planning performance by integrating world knowledge, offering a promising approach for future research in agent planning.This paper introduces a parametric World Knowledge Model (WKM) to enhance agent planning by incorporating global and local knowledge. The WKM is designed to provide prior task knowledge for global planning and dynamic state knowledge for local planning, addressing the limitations of large language models (LLMs) in understanding the physical world. The WKM is trained using knowledge synthesized from expert and sampled trajectories, enabling the agent to self-synthesize task knowledge and summarize state knowledge for each planning step. The agent model is then retrained to adapt to the task knowledge, with both the agent and knowledge models trained using LoRA. During planning, the WKM provides global prior knowledge and maintains dynamic state knowledge, guiding the agent to avoid blind trial-and-error and hallucinatory actions. The agent uses the WKM to retrieve relevant state knowledge from a pre-built knowledge base, combining it with the agent's own predictions to make informed decisions. Experimental results on three real-world simulated datasets (ALFWorld, WebShop, and ScienceWorld) with three state-of-the-art open-source LLMs (Mistral-7B, Gemma-7B, and Llama-3-8B) show that the proposed method outperforms various strong baselines. The WKM effectively reduces blind trial-and-error and hallucinatory actions, generalizes better to unseen tasks, and demonstrates the feasibility of weak-guide-strong planning. Additionally, the WKM shows potential for unified training across multiple tasks and highlights the importance of implicit knowledge constraints in agent planning. The results indicate that the WKM significantly improves agent planning performance by integrating world knowledge, offering a promising approach for future research in agent planning.
Reach us at info@study.space
[slides and audio] Agent Planning with World Knowledge Model