This tutorial article provides a comprehensive overview of offline reinforcement learning (RL), which involves using previously collected data to train RL algorithms without additional online data collection. The authors highlight the potential of offline RL to turn large datasets into powerful decision-making engines, particularly in domains such as healthcare, education, and robotics. However, the challenges of offline RL are significant, including the inability to improve exploration and the need to handle distributional shift. The article covers the problem formulation, key concepts, and various offline RL methods, including off-policy evaluation via importance sampling, Q-learning, actor-critic algorithms, and model-based approaches. It also discusses the limitations of current methods and explores potential solutions, applications, and open problems in the field. The authors aim to provide readers with the conceptual tools needed to start research in offline RL, addressing the unique challenges and opportunities in this area.This tutorial article provides a comprehensive overview of offline reinforcement learning (RL), which involves using previously collected data to train RL algorithms without additional online data collection. The authors highlight the potential of offline RL to turn large datasets into powerful decision-making engines, particularly in domains such as healthcare, education, and robotics. However, the challenges of offline RL are significant, including the inability to improve exploration and the need to handle distributional shift. The article covers the problem formulation, key concepts, and various offline RL methods, including off-policy evaluation via importance sampling, Q-learning, actor-critic algorithms, and model-based approaches. It also discusses the limitations of current methods and explores potential solutions, applications, and open problems in the field. The authors aim to provide readers with the conceptual tools needed to start research in offline RL, addressing the unique challenges and opportunities in this area.