Cooperative multi-agent learning involves multiple agents working together to solve tasks or maximize utility. Due to interactions among agents, problem complexity increases rapidly with the number of agents or their behavioral sophistication. This has led to increased interest in machine learning techniques to automate the search and optimization process. This survey provides a broad overview of cooperative multi-agent learning, covering areas such as reinforcement learning, evolutionary computation, game theory, complex systems, agent modeling, and robotics. It divides the work into two categories: team learning, where a single learner discovers joint solutions, and concurrent learning, where multiple learners, often one per agent, are used. The survey also discusses direct and indirect communication, task decomposition, scalability, and adaptive dynamics. It concludes with a presentation of multi-agent learning problem domains and resources. Multi-agent systems (MAS) involve agents with autonomy and interactions. Machine learning automates the inductive process of discovering solutions. Cooperative multi-agent learning focuses on agents working together rather than competing. The survey discusses various machine learning methods, including supervised, unsupervised, and reward-based learning. Reinforcement learning and evolutionary computation are particularly relevant. The survey also covers team learning, where a single learner improves team behavior, and concurrent learning, where multiple learners adapt in the context of others. Team learning has challenges such as large state spaces, while concurrent learning requires new machine learning methods due to non-stationary environments. Team learning involves a single learner discovering behaviors for a team of agents. It is simpler than concurrent learning but may have scalability issues. Team learning can be homogeneous (all agents have the same behavior) or heterogeneous (each agent has unique behavior). Hybrid team learning splits agents into squads with shared behaviors. Heterogeneous team learning can yield better solutions through specialization but requires larger search spaces. Experiments show that homogeneous learning may be suitable for tasks like foraging, while heterogeneous learning is better for tasks requiring specialization. Concurrent learning involves multiple learners, each learning for a team member. It allows for more flexibility but introduces challenges such as co-adaptation and non-stationary environments. Credit assignment is a key issue in concurrent learning, as rewards must be distributed among learners. Global reward assigns rewards equally, while local reward focuses on individual performance. Credit assignment methods impact learning dynamics and can lead to different outcomes depending on the scenario. The dynamics of learning in multi-agent systems are complex, with challenges such as adapting to changing environments and ensuring cooperation. Evolutionary game theory and other tools help analyze these dynamics, but scalability remains a challenge. The survey highlights the importance of credit assignment, team learning, and concurrent learning in cooperative multi-agent systems.Cooperative multi-agent learning involves multiple agents working together to solve tasks or maximize utility. Due to interactions among agents, problem complexity increases rapidly with the number of agents or their behavioral sophistication. This has led to increased interest in machine learning techniques to automate the search and optimization process. This survey provides a broad overview of cooperative multi-agent learning, covering areas such as reinforcement learning, evolutionary computation, game theory, complex systems, agent modeling, and robotics. It divides the work into two categories: team learning, where a single learner discovers joint solutions, and concurrent learning, where multiple learners, often one per agent, are used. The survey also discusses direct and indirect communication, task decomposition, scalability, and adaptive dynamics. It concludes with a presentation of multi-agent learning problem domains and resources. Multi-agent systems (MAS) involve agents with autonomy and interactions. Machine learning automates the inductive process of discovering solutions. Cooperative multi-agent learning focuses on agents working together rather than competing. The survey discusses various machine learning methods, including supervised, unsupervised, and reward-based learning. Reinforcement learning and evolutionary computation are particularly relevant. The survey also covers team learning, where a single learner improves team behavior, and concurrent learning, where multiple learners adapt in the context of others. Team learning has challenges such as large state spaces, while concurrent learning requires new machine learning methods due to non-stationary environments. Team learning involves a single learner discovering behaviors for a team of agents. It is simpler than concurrent learning but may have scalability issues. Team learning can be homogeneous (all agents have the same behavior) or heterogeneous (each agent has unique behavior). Hybrid team learning splits agents into squads with shared behaviors. Heterogeneous team learning can yield better solutions through specialization but requires larger search spaces. Experiments show that homogeneous learning may be suitable for tasks like foraging, while heterogeneous learning is better for tasks requiring specialization. Concurrent learning involves multiple learners, each learning for a team member. It allows for more flexibility but introduces challenges such as co-adaptation and non-stationary environments. Credit assignment is a key issue in concurrent learning, as rewards must be distributed among learners. Global reward assigns rewards equally, while local reward focuses on individual performance. Credit assignment methods impact learning dynamics and can lead to different outcomes depending on the scenario. The dynamics of learning in multi-agent systems are complex, with challenges such as adapting to changing environments and ensuring cooperation. Evolutionary game theory and other tools help analyze these dynamics, but scalability remains a challenge. The survey highlights the importance of credit assignment, team learning, and concurrent learning in cooperative multi-agent systems.