This paper introduces a framework for prioritizing experience replay in reinforcement learning, aiming to improve the efficiency and effectiveness of learning. The authors propose a method to prioritize transitions based on their expected learning progress, measured by the magnitude of temporal-difference (TD) errors. This approach allows for more frequent replay of important transitions, reducing the overall number of updates required for learning. The prioritized replay is applied to Deep Q-Networks (DQN), a reinforcement learning algorithm that achieved human-level performance on many Atari games. The results show that prioritized replay outperforms uniform replay on 41 out of 49 games, achieving a new state-of-the-art performance. The paper also discusses the implementation details, including stochastic prioritization to address diversity and bias correction using importance sampling. The authors further explore extensions of prioritized replay to supervised learning and off-policy reinforcement learning, highlighting its potential benefits in these areas.This paper introduces a framework for prioritizing experience replay in reinforcement learning, aiming to improve the efficiency and effectiveness of learning. The authors propose a method to prioritize transitions based on their expected learning progress, measured by the magnitude of temporal-difference (TD) errors. This approach allows for more frequent replay of important transitions, reducing the overall number of updates required for learning. The prioritized replay is applied to Deep Q-Networks (DQN), a reinforcement learning algorithm that achieved human-level performance on many Atari games. The results show that prioritized replay outperforms uniform replay on 41 out of 49 games, achieving a new state-of-the-art performance. The paper also discusses the implementation details, including stochastic prioritization to address diversity and bias correction using importance sampling. The authors further explore extensions of prioritized replay to supervised learning and off-policy reinforcement learning, highlighting its potential benefits in these areas.