This paper addresses the challenges of catastrophic forgetting and loss of plasticity in continual learning, particularly in the context of streaming learning. The authors introduce Utility-based Perturbed Gradient Descent (UPGD), a novel approach that combines gradient updates with perturbations to protect useful units from forgetting and rejuvenate less useful units. UPGD is designed to handle both issues simultaneously, which is a significant gap in existing methods. The paper evaluates UPGD on challenging streaming learning setups with hundreds of non-stationarities and unknown task boundaries, demonstrating its effectiveness in maintaining performance and surpassing or competing with other methods. Additionally, UPGD is shown to avoid the performance drop observed in Adam during extended reinforcement learning experiments with PPO, further validating its ability to address both catastrophic forgetting and loss of plasticity. The paper also introduces metrics for evaluating plasticity and forgetting, providing a comprehensive framework for understanding and addressing these issues in continual learning.This paper addresses the challenges of catastrophic forgetting and loss of plasticity in continual learning, particularly in the context of streaming learning. The authors introduce Utility-based Perturbed Gradient Descent (UPGD), a novel approach that combines gradient updates with perturbations to protect useful units from forgetting and rejuvenate less useful units. UPGD is designed to handle both issues simultaneously, which is a significant gap in existing methods. The paper evaluates UPGD on challenging streaming learning setups with hundreds of non-stationarities and unknown task boundaries, demonstrating its effectiveness in maintaining performance and surpassing or competing with other methods. Additionally, UPGD is shown to avoid the performance drop observed in Adam during extended reinforcement learning experiments with PPO, further validating its ability to address both catastrophic forgetting and loss of plasticity. The paper also introduces metrics for evaluating plasticity and forgetting, providing a comprehensive framework for understanding and addressing these issues in continual learning.