This paper introduces PackNet, a method to add multiple tasks to a single deep neural network while avoiding catastrophic forgetting. Inspired by network pruning techniques, PackNet exploits redundancies in large deep networks to free up parameters that can then be used to learn new tasks. By performing iterative pruning and retraining, PackNet sequentially "packs" multiple tasks into a single network while maintaining minimal performance drop and storage overhead. Unlike prior methods that use proxy losses to maintain accuracy on older tasks, PackNet directly optimizes for the current task. The method is tested on various network architectures and large-scale datasets, showing improved robustness against catastrophic forgetting compared to prior work. Specifically, PackNet adds three fine-grained classification tasks to a single ImageNet-trained VGG-16 network, achieving accuracies close to those of separately trained networks for each task. The method is also shown to outperform joint training when adding the Places365 dataset to an ImageNet-trained network, and performs well on a range of architectures including VGG-16 with batch normalization, ResNets, and DenseNets. PackNet's approach involves pruning redundant parameters across all layers of a deep network after it has been trained for a task, keeping the surviving parameters fixed while using the freed-up parameters to learn new tasks. This process is repeated for adding multiple tasks, with the task-specific parameter masks generated by pruning allowing the models to maintain the same level of accuracy even after adding multiple tasks. The method's effectiveness is demonstrated through extensive experiments on various tasks where high-level feature transfer does not perform well, indicating the need to modify parameters at all layers. The results show that PackNet significantly outperforms prior work in terms of robustness to catastrophic forgetting and the number and complexity of added tasks. The method is also shown to be superior to joint training when adding the Places365 dataset to an ImageNet-trained network, and achieves competitive performance on a broad range of architectures. The paper also discusses related work, including methods like Learning without Forgetting (LwF) and Elastic Weight Consolidation (EWC), and compares PackNet's approach with these methods. The results show that PackNet's method is more effective in maintaining performance on older tasks while adding new tasks, and that it achieves much lower storage overhead compared to other methods. The paper concludes that PackNet's approach is effective for adding multiple tasks to a single network while maintaining performance on prior tasks, and that it works well for both large and compact networks.This paper introduces PackNet, a method to add multiple tasks to a single deep neural network while avoiding catastrophic forgetting. Inspired by network pruning techniques, PackNet exploits redundancies in large deep networks to free up parameters that can then be used to learn new tasks. By performing iterative pruning and retraining, PackNet sequentially "packs" multiple tasks into a single network while maintaining minimal performance drop and storage overhead. Unlike prior methods that use proxy losses to maintain accuracy on older tasks, PackNet directly optimizes for the current task. The method is tested on various network architectures and large-scale datasets, showing improved robustness against catastrophic forgetting compared to prior work. Specifically, PackNet adds three fine-grained classification tasks to a single ImageNet-trained VGG-16 network, achieving accuracies close to those of separately trained networks for each task. The method is also shown to outperform joint training when adding the Places365 dataset to an ImageNet-trained network, and performs well on a range of architectures including VGG-16 with batch normalization, ResNets, and DenseNets. PackNet's approach involves pruning redundant parameters across all layers of a deep network after it has been trained for a task, keeping the surviving parameters fixed while using the freed-up parameters to learn new tasks. This process is repeated for adding multiple tasks, with the task-specific parameter masks generated by pruning allowing the models to maintain the same level of accuracy even after adding multiple tasks. The method's effectiveness is demonstrated through extensive experiments on various tasks where high-level feature transfer does not perform well, indicating the need to modify parameters at all layers. The results show that PackNet significantly outperforms prior work in terms of robustness to catastrophic forgetting and the number and complexity of added tasks. The method is also shown to be superior to joint training when adding the Places365 dataset to an ImageNet-trained network, and achieves competitive performance on a broad range of architectures. The paper also discusses related work, including methods like Learning without Forgetting (LwF) and Elastic Weight Consolidation (EWC), and compares PackNet's approach with these methods. The results show that PackNet's method is more effective in maintaining performance on older tasks while adding new tasks, and that it achieves much lower storage overhead compared to other methods. The paper concludes that PackNet's approach is effective for adding multiple tasks to a single network while maintaining performance on prior tasks, and that it works well for both large and compact networks.