The paper introduces CLEAR (Continual Learning with Experience And Replay), a method designed to address the challenge of catastrophic forgetting in multi-task reinforcement learning. CLEAR combines on-policy learning from new experiences and off-policy learning from replayed experiences to enhance stability and plasticity. The method also includes behavioral cloning to maintain stability by ensuring that the current policy remains close to its historical self. The authors demonstrate that CLEAR outperforms state-of-the-art techniques like Elastic Weight Consolidation and Progress & Compress, while being simpler and not requiring knowledge of task boundaries. Experiments show that CLEAR effectively reduces catastrophic forgetting, preserving performance on past tasks while rapidly adapting to new ones. The method is shown to be robust even with limited memory, making it a practical solution for real-world applications.The paper introduces CLEAR (Continual Learning with Experience And Replay), a method designed to address the challenge of catastrophic forgetting in multi-task reinforcement learning. CLEAR combines on-policy learning from new experiences and off-policy learning from replayed experiences to enhance stability and plasticity. The method also includes behavioral cloning to maintain stability by ensuring that the current policy remains close to its historical self. The authors demonstrate that CLEAR outperforms state-of-the-art techniques like Elastic Weight Consolidation and Progress & Compress, while being simpler and not requiring knowledge of task boundaries. Experiments show that CLEAR effectively reduces catastrophic forgetting, preserving performance on past tasks while rapidly adapting to new ones. The method is shown to be robust even with limited memory, making it a practical solution for real-world applications.