This report presents a unified approach for the study of constrained Markov decision processes (CMDPs) with a countable state space and unbounded costs. It considers a single controller with multiple objectives, aiming to design a controller that minimizes one cost objective while satisfying inequality constraints on others. The objectives studied include expected average cost and expected total cost (including discounted cost as a special case). Two frameworks are introduced: one where costs are bounded below and another called the contracting framework. The report characterizes the set of achievable expected occupation measures and performance vectors, reducing the original control problem to an infinite linear program. A Lagrangian approach is presented to obtain sensitivity analysis, yielding asymptotic results for constrained control problems, including convergence of value and policies in time horizon and discount factor. Several state truncation algorithms are introduced to approximate the original control problem using finite linear programs. The report also discusses various types of CMDPs, solution approaches, and the convex analytical approach using occupation measures. It covers the total cost, expected average cost, and their respective linear programming and Lagrangian formulations. The analysis includes sensitivity analysis, convergence properties, and state truncation techniques for CMDPs. The report concludes with a detailed discussion of the theoretical foundations, solution methods, and practical applications of constrained Markov decision processes.This report presents a unified approach for the study of constrained Markov decision processes (CMDPs) with a countable state space and unbounded costs. It considers a single controller with multiple objectives, aiming to design a controller that minimizes one cost objective while satisfying inequality constraints on others. The objectives studied include expected average cost and expected total cost (including discounted cost as a special case). Two frameworks are introduced: one where costs are bounded below and another called the contracting framework. The report characterizes the set of achievable expected occupation measures and performance vectors, reducing the original control problem to an infinite linear program. A Lagrangian approach is presented to obtain sensitivity analysis, yielding asymptotic results for constrained control problems, including convergence of value and policies in time horizon and discount factor. Several state truncation algorithms are introduced to approximate the original control problem using finite linear programs. The report also discusses various types of CMDPs, solution approaches, and the convex analytical approach using occupation measures. It covers the total cost, expected average cost, and their respective linear programming and Lagrangian formulations. The analysis includes sensitivity analysis, convergence properties, and state truncation techniques for CMDPs. The report concludes with a detailed discussion of the theoretical foundations, solution methods, and practical applications of constrained Markov decision processes.