The paper introduces *Progressive Networks*, a novel model architecture designed to address the challenge of learning complex sequences of tasks while leveraging transfer and avoiding catastrophic forgetting. Progressive Networks retain a pool of pre-trained models throughout training and learn lateral connections to extract useful features for new tasks, achieving richer compositionality and integrating prior knowledge at each layer of the feature hierarchy. The authors evaluate this architecture extensively on various reinforcement learning tasks, including Atari games and 3D maze games, demonstrating superior performance compared to common baselines based on pretraining and fine-tuning. They also develop a novel analysis using Fisher Information and perturbation to detail how and where transfer occurs across tasks. The results show that Progressive Networks can effectively exploit transfer for compatible source and target domains, robustly handle harmful features from incompatible tasks, and increase positive transfer with the number of columns, suggesting a constructive rather than destructive nature.The paper introduces *Progressive Networks*, a novel model architecture designed to address the challenge of learning complex sequences of tasks while leveraging transfer and avoiding catastrophic forgetting. Progressive Networks retain a pool of pre-trained models throughout training and learn lateral connections to extract useful features for new tasks, achieving richer compositionality and integrating prior knowledge at each layer of the feature hierarchy. The authors evaluate this architecture extensively on various reinforcement learning tasks, including Atari games and 3D maze games, demonstrating superior performance compared to common baselines based on pretraining and fine-tuning. They also develop a novel analysis using Fisher Information and perturbation to detail how and where transfer occurs across tasks. The results show that Progressive Networks can effectively exploit transfer for compatible source and target domains, robustly handle harmful features from incompatible tasks, and increase positive transfer with the number of columns, suggesting a constructive rather than destructive nature.