This paper explores the integration of Large Language Models (LLMs) into Multi-Agent Reinforcement Learning (MARL) and discusses current research and future directions. LLMs have shown strong performance in various tasks, including question answering, arithmetic, and poetry. While research has shown that LLMs can be used in RL, extending this to multi-agent systems is challenging due to issues like coordination and communication. The paper surveys existing LLM-based single-agent and multi-agent RL frameworks and highlights potential research directions, focusing on cooperative tasks with shared goals and communication among agents. It also considers human-in-the-loop scenarios enabled by the language component in the framework. MARL has emerged as a popular approach for addressing coordination in multi-agent systems. Unlike traditional methods, MARL improves scalability and robustness to uncertainty and dynamicity. However, communication and coordination among agents remain key challenges. Recent advances in Natural Language Processing (NLP) have enabled language-conditioned MARL, which is a promising research area. NLP has been a long-standing research topic, with many models developed for language modeling. The integration of NLP with single-agent RL has led to the development of language-conditioned RL frameworks, especially with the rise of LLMs. Pre-trained LLMs contain general human knowledge and can adapt to RL problems without retraining. This integration allows for dynamic adjustment of agent behaviors based on linguistic input. LLMs can generate new information based on few examples, which is valuable in multi-agent systems where agents must coordinate and cooperate based on shared goals. The paper discusses existing LLM-based MARL frameworks, including those for problem-solving and embodied applications. These frameworks demonstrate enhanced performance and the potential to augment models with reasoning techniques. The paper also highlights challenges and future research directions, including personality-enabled cooperation, language-enabled human-in-the-loop frameworks, traditional MARL and LLM co-design, and safety and security in multi-agent systems. Overall, the paper argues that language-conditioned MARL holds significant promise for advancing the capabilities of multi-agent systems. Using natural language, these systems can achieve higher levels of coordination and understanding, which is essential for complex environments. The paper concludes that LLMs offer a new approach to designing MARL frameworks, more akin to modeling the group learning process of animals or humans, where knowledge is transferred or exchanged via natural languages.This paper explores the integration of Large Language Models (LLMs) into Multi-Agent Reinforcement Learning (MARL) and discusses current research and future directions. LLMs have shown strong performance in various tasks, including question answering, arithmetic, and poetry. While research has shown that LLMs can be used in RL, extending this to multi-agent systems is challenging due to issues like coordination and communication. The paper surveys existing LLM-based single-agent and multi-agent RL frameworks and highlights potential research directions, focusing on cooperative tasks with shared goals and communication among agents. It also considers human-in-the-loop scenarios enabled by the language component in the framework. MARL has emerged as a popular approach for addressing coordination in multi-agent systems. Unlike traditional methods, MARL improves scalability and robustness to uncertainty and dynamicity. However, communication and coordination among agents remain key challenges. Recent advances in Natural Language Processing (NLP) have enabled language-conditioned MARL, which is a promising research area. NLP has been a long-standing research topic, with many models developed for language modeling. The integration of NLP with single-agent RL has led to the development of language-conditioned RL frameworks, especially with the rise of LLMs. Pre-trained LLMs contain general human knowledge and can adapt to RL problems without retraining. This integration allows for dynamic adjustment of agent behaviors based on linguistic input. LLMs can generate new information based on few examples, which is valuable in multi-agent systems where agents must coordinate and cooperate based on shared goals. The paper discusses existing LLM-based MARL frameworks, including those for problem-solving and embodied applications. These frameworks demonstrate enhanced performance and the potential to augment models with reasoning techniques. The paper also highlights challenges and future research directions, including personality-enabled cooperation, language-enabled human-in-the-loop frameworks, traditional MARL and LLM co-design, and safety and security in multi-agent systems. Overall, the paper argues that language-conditioned MARL holds significant promise for advancing the capabilities of multi-agent systems. Using natural language, these systems can achieve higher levels of coordination and understanding, which is essential for complex environments. The paper concludes that LLMs offer a new approach to designing MARL frameworks, more akin to modeling the group learning process of animals or humans, where knowledge is transferred or exchanged via natural languages.