Gradient Episodic Memory (GEM) is a model for continual learning that addresses the problem of catastrophic forgetting, where a model loses previously learned skills when learning new tasks. The paper introduces a framework for continual learning where a model observes a sequence of tasks, each with its own input and target data, and learns to predict targets for unseen examples. The model evaluates performance using metrics that measure both forward and backward transfer, which assess how well the model can learn new tasks while retaining knowledge from previous ones. The paper proposes GEM, which uses an episodic memory to store examples from previous tasks, allowing the model to retain knowledge and transfer it to new tasks. GEM minimizes negative backward transfer (catastrophic forgetting) by ensuring that learning new tasks does not degrade performance on previously learned tasks. The model is trained using a quadratic programming approach to optimize the learning process while maintaining performance on past tasks. Experiments on variants of the MNIST and CIFAR-100 datasets show that GEM outperforms state-of-the-art methods in continual learning. The model achieves higher accuracy and better transfer performance compared to alternatives such as EWC and iCARL. GEM is efficient in terms of computation, as it optimizes over a number of variables equal to the number of tasks rather than the number of parameters in the neural network. The paper also discusses the importance of memory, the number of training passes, and the order of tasks in continual learning, showing that memory-based methods like GEM perform better in minimizing negative backward transfer. The paper concludes that GEM is a promising approach for continual learning, offering a balance between performance and computational efficiency. It highlights the need for further research in this area, particularly in leveraging structured task descriptors and improving memory management strategies.Gradient Episodic Memory (GEM) is a model for continual learning that addresses the problem of catastrophic forgetting, where a model loses previously learned skills when learning new tasks. The paper introduces a framework for continual learning where a model observes a sequence of tasks, each with its own input and target data, and learns to predict targets for unseen examples. The model evaluates performance using metrics that measure both forward and backward transfer, which assess how well the model can learn new tasks while retaining knowledge from previous ones. The paper proposes GEM, which uses an episodic memory to store examples from previous tasks, allowing the model to retain knowledge and transfer it to new tasks. GEM minimizes negative backward transfer (catastrophic forgetting) by ensuring that learning new tasks does not degrade performance on previously learned tasks. The model is trained using a quadratic programming approach to optimize the learning process while maintaining performance on past tasks. Experiments on variants of the MNIST and CIFAR-100 datasets show that GEM outperforms state-of-the-art methods in continual learning. The model achieves higher accuracy and better transfer performance compared to alternatives such as EWC and iCARL. GEM is efficient in terms of computation, as it optimizes over a number of variables equal to the number of tasks rather than the number of parameters in the neural network. The paper also discusses the importance of memory, the number of training passes, and the order of tasks in continual learning, showing that memory-based methods like GEM perform better in minimizing negative backward transfer. The paper concludes that GEM is a promising approach for continual learning, offering a balance between performance and computational efficiency. It highlights the need for further research in this area, particularly in leveraging structured task descriptors and improving memory management strategies.