2 Apr 2024 | Sihao Hu†, Tiansheng Huang†, Fatih İlhan†, Selim Tekin†, Gaowen Liu†, Ramana Kompella‡, Ling Liu†
This paper provides a comprehensive overview of Large Language Model (LLM)-based game agents, focusing on their conceptual architecture, existing literature, and future research directions. The conceptual architecture of LLM-based game agents is centered around six essential functional components: perception, memory, thinking, role-playing, action, and learning. The paper surveys existing LLM-based game agents across six game genres—adventure, communication, competition, cooperation, simulation, and crafting & exploration—describing their methodologies and adaptation agility. It highlights the challenges and strategies used in each genre, such as the use of external visual encoders, multimodal LLMs, and reinforcement learning. The paper also presents an outlook on future research directions and maintains a curated list of relevant papers for further exploration. The evaluation metrics for game agents vary across different games, with task success rates, win rates, game scores, and Elo ratings being common metrics. The paper aims to catalyze progress in this nascent research area and inspire further innovation in LLM-based game agents.This paper provides a comprehensive overview of Large Language Model (LLM)-based game agents, focusing on their conceptual architecture, existing literature, and future research directions. The conceptual architecture of LLM-based game agents is centered around six essential functional components: perception, memory, thinking, role-playing, action, and learning. The paper surveys existing LLM-based game agents across six game genres—adventure, communication, competition, cooperation, simulation, and crafting & exploration—describing their methodologies and adaptation agility. It highlights the challenges and strategies used in each genre, such as the use of external visual encoders, multimodal LLMs, and reinforcement learning. The paper also presents an outlook on future research directions and maintains a curated list of relevant papers for further exploration. The evaluation metrics for game agents vary across different games, with task success rates, win rates, game scores, and Elo ratings being common metrics. The paper aims to catalyze progress in this nascent research area and inspire further innovation in LLM-based game agents.