25 Jan 2017 | James Kirkpatrick, Razvan Pascanu, Neil Rabinowitz, Joel Veness, Guillaume Desjardins, Andrei A. Rusu, Kieran Milan, John Quan, Tiago Ramalho, Agnieszka Grabska-Barwinska, Demis Hassabis, Claudia Clopath, Dharshan Kumaran, and Raia Hadsell
The paper addresses the challenge of catastrophic forgetting in neural networks, which is the tendency for a network to forget previously learned tasks when learning new ones. The authors propose a novel algorithm called Elastic Weight Consolidation (EWC), which selectively slows down learning on weights important for previously learned tasks, thereby preserving their knowledge. EWC is inspired by neurobiological models of synaptic consolidation, where synapses that are crucial for a task are protected from forgetting. The algorithm is implemented as a quadratic penalty that pulls weights back towards their old values, with the strength of the penalty determined by the importance of the weights to previously learned tasks. The authors demonstrate that EWC can effectively support continual learning in both supervised learning tasks (using the MNIST dataset) and reinforcement learning tasks (using Atari 2600 games). They show that EWC allows networks to learn multiple tasks sequentially without forgetting older ones, outperforming traditional methods that rely on hyperparameter tuning and regularization techniques. The paper also discusses the computational complexity and limitations of EWC, noting that it is efficient due to approximations used in its implementation but suggests potential improvements using Bayesian neural networks. Overall, the work provides a strong foundation for further research into continual learning in artificial intelligence.The paper addresses the challenge of catastrophic forgetting in neural networks, which is the tendency for a network to forget previously learned tasks when learning new ones. The authors propose a novel algorithm called Elastic Weight Consolidation (EWC), which selectively slows down learning on weights important for previously learned tasks, thereby preserving their knowledge. EWC is inspired by neurobiological models of synaptic consolidation, where synapses that are crucial for a task are protected from forgetting. The algorithm is implemented as a quadratic penalty that pulls weights back towards their old values, with the strength of the penalty determined by the importance of the weights to previously learned tasks. The authors demonstrate that EWC can effectively support continual learning in both supervised learning tasks (using the MNIST dataset) and reinforcement learning tasks (using Atari 2600 games). They show that EWC allows networks to learn multiple tasks sequentially without forgetting older ones, outperforming traditional methods that rely on hyperparameter tuning and regularization techniques. The paper also discusses the computational complexity and limitations of EWC, noting that it is efficient due to approximations used in its implementation but suggests potential improvements using Bayesian neural networks. Overall, the work provides a strong foundation for further research into continual learning in artificial intelligence.