July 6, 2015 | Frans A. Oliehoek & Christopher Amato
This chapter introduces the concept of decentralized partially observable Markov decision processes (Dec-POMDPs) and their application in multiagent systems under uncertainty. It begins with motivating examples, such as multi-robot coordination and efficient sensor networks, to illustrate the challenges and uncertainties in automated decision-making in real-world decentralized systems. The chapter then delves into the formal definition of Dec-POMDPs, which extend the partially observable Markov decision process (POMDP) framework to include multiple agents. It covers the single-agent decision frameworks of Markov decision processes (MDPs) and POMDPs, providing a background for understanding the multiagent extension. The chapter also discusses the importance of uncertainty in multiagent systems, including outcome uncertainty, state uncertainty, and uncertainty with respect to other agents. Finally, it explores various application domains where Dec-POMDPs are particularly useful, such as distributed load balancing, transmission protocols, sensor networks, traffic light control, and cooperative robotics.This chapter introduces the concept of decentralized partially observable Markov decision processes (Dec-POMDPs) and their application in multiagent systems under uncertainty. It begins with motivating examples, such as multi-robot coordination and efficient sensor networks, to illustrate the challenges and uncertainties in automated decision-making in real-world decentralized systems. The chapter then delves into the formal definition of Dec-POMDPs, which extend the partially observable Markov decision process (POMDP) framework to include multiple agents. It covers the single-agent decision frameworks of Markov decision processes (MDPs) and POMDPs, providing a background for understanding the multiagent extension. The chapter also discusses the importance of uncertainty in multiagent systems, including outcome uncertainty, state uncertainty, and uncertainty with respect to other agents. Finally, it explores various application domains where Dec-POMDPs are particularly useful, such as distributed load balancing, transmission protocols, sensor networks, traffic light control, and cooperative robotics.