The Arcade Learning Environment (ALE) is a platform for evaluating general AI agents by providing access to hundreds of Atari 2600 game environments. It presents significant challenges for reinforcement learning, model learning, planning, imitation learning, transfer learning, and intrinsic motivation. ALE allows for the development and benchmarking of domain-independent agents using well-established AI techniques. The environment includes a software framework for interfacing with emulated Atari 2600 games, enabling the study of planning and reinforcement learning. ALE provides a rigorous testbed for evaluating and comparing approaches to these problems. The paper describes the ALE framework, its interface with Atari 2600 games, and the results of benchmarking various reinforcement learning and planning methods on a wide range of games. The results show that while some learning progress is possible in Atari 2600 games, much more work remains to be done. Different methods perform well on different games, and no single method performs well on all games. The paper also introduces evaluation metrics for general Atari 2600 agents, including normalized scores, average and median scores, and score distributions. These metrics allow for the comparison of agents across a diverse set of domains. The paper concludes that ALE provides a valuable platform for evaluating general AI agents and highlights the importance of using empirical methods to assess their performance.The Arcade Learning Environment (ALE) is a platform for evaluating general AI agents by providing access to hundreds of Atari 2600 game environments. It presents significant challenges for reinforcement learning, model learning, planning, imitation learning, transfer learning, and intrinsic motivation. ALE allows for the development and benchmarking of domain-independent agents using well-established AI techniques. The environment includes a software framework for interfacing with emulated Atari 2600 games, enabling the study of planning and reinforcement learning. ALE provides a rigorous testbed for evaluating and comparing approaches to these problems. The paper describes the ALE framework, its interface with Atari 2600 games, and the results of benchmarking various reinforcement learning and planning methods on a wide range of games. The results show that while some learning progress is possible in Atari 2600 games, much more work remains to be done. Different methods perform well on different games, and no single method performs well on all games. The paper also introduces evaluation metrics for general Atari 2600 agents, including normalized scores, average and median scores, and score distributions. These metrics allow for the comparison of agents across a diverse set of domains. The paper concludes that ALE provides a valuable platform for evaluating general AI agents and highlights the importance of using empirical methods to assess their performance.