This paper presents an overview of decision-theoretic planning (DTP) and Markov decision processes (MDPs), showing how they provide a unifying framework for modeling planning problems in AI. It discusses the structural properties of MDPs that can be exploited to construct optimal or approximately optimal policies. The paper also surveys various representations and algorithms for classical and decision-theoretic planning problems, focusing on abstraction, aggregation, and decomposition techniques. It emphasizes the importance of structural assumptions in MDPs and how they can be used to improve computational efficiency. The paper also addresses the challenges of applying MDPs to AI planning, including the need for efficient representations and algorithms that can handle the complexity of planning problems. It concludes by highlighting the potential of MDPs in AI planning and the need for further research in this area.This paper presents an overview of decision-theoretic planning (DTP) and Markov decision processes (MDPs), showing how they provide a unifying framework for modeling planning problems in AI. It discusses the structural properties of MDPs that can be exploited to construct optimal or approximately optimal policies. The paper also surveys various representations and algorithms for classical and decision-theoretic planning problems, focusing on abstraction, aggregation, and decomposition techniques. It emphasizes the importance of structural assumptions in MDPs and how they can be used to improve computational efficiency. The paper also addresses the challenges of applying MDPs to AI planning, including the need for efficient representations and algorithms that can handle the complexity of planning problems. It concludes by highlighting the potential of MDPs in AI planning and the need for further research in this area.