The paper introduces D4RL, a suite of benchmarks designed for offline reinforcement learning (RL), where policies are learned from static datasets. The benchmarks are tailored to reflect real-world applications, focusing on tasks and data collection strategies that exercise key properties such as partial observability, passively logged data, and human demonstrations. The authors highlight the limitations of existing benchmarks, which often use data from online RL training runs, and argue that these do not adequately represent the challenges of offline RL. D4RL includes a variety of tasks and datasets, such as Maze2D, AntMaze, Gym-MuJoCo, Adroit, FrankaKitchen, Flow, and Offline CARLA, each designed to test different aspects of offline RL algorithms. The paper also provides an evaluation protocol and open-source implementations to facilitate research and comparison across algorithms. The results show that current algorithms struggle with challenging properties like narrow data distributions, sparse rewards, and non-representable policies, highlighting the need for more robust methods in offline RL. The authors conclude by discussing the potential of offline RL in leveraging large, previously collected datasets and the importance of standardized benchmarks for future progress.The paper introduces D4RL, a suite of benchmarks designed for offline reinforcement learning (RL), where policies are learned from static datasets. The benchmarks are tailored to reflect real-world applications, focusing on tasks and data collection strategies that exercise key properties such as partial observability, passively logged data, and human demonstrations. The authors highlight the limitations of existing benchmarks, which often use data from online RL training runs, and argue that these do not adequately represent the challenges of offline RL. D4RL includes a variety of tasks and datasets, such as Maze2D, AntMaze, Gym-MuJoCo, Adroit, FrankaKitchen, Flow, and Offline CARLA, each designed to test different aspects of offline RL algorithms. The paper also provides an evaluation protocol and open-source implementations to facilitate research and comparison across algorithms. The results show that current algorithms struggle with challenging properties like narrow data distributions, sparse rewards, and non-representable policies, highlighting the need for more robust methods in offline RL. The authors conclude by discussing the potential of offline RL in leveraging large, previously collected datasets and the importance of standardized benchmarks for future progress.