This paper investigates the causes of plasticity loss in neural networks, which refers to the network's reduced ability to adapt to new information during training. The assumption that data is stationary is often violated in settings like deep reinforcement learning, leading to unstable and brittle learning algorithms. Plasticity loss is attributed to multiple independent mechanisms, and while addressing a single mechanism is insufficient, combining interventions on multiple mechanisms can significantly improve robustness. The study identifies that plasticity loss can be decomposed into several mechanisms, including preactivation distribution shift, regression target magnitude, and parameter growth. It shows that layer normalization and weight decay are effective in maintaining plasticity across various synthetic and real-world nonstationary tasks, including reinforcement learning in the Arcade Learning Environment. The paper also explores the empirical neural tangent kernel (NTK) as a diagnostic tool for plasticity loss, revealing that networks with plasticity loss exhibit similar pathologies in their NTKs. This suggests that the NTK can be used to predict and diagnose training difficulties. The study proposes a "Swiss cheese model" of mitigation strategies, where interventions targeting different mechanisms can be combined to achieve additive benefits. It demonstrates that combining layer normalization with L2 regularization is highly effective in maintaining plasticity in both synthetic and natural distribution shift scenarios. The paper concludes that while no single network property can explain all instances of plasticity loss, several independent mechanisms are responsible for a large fraction of observed cases. By identifying and combining effective interventions for each mechanism, the combinatorial complexity of mitigation strategies can be significantly reduced. This approach offers promising directions for future research in stabilizing optimization in nonstationary learning problems.This paper investigates the causes of plasticity loss in neural networks, which refers to the network's reduced ability to adapt to new information during training. The assumption that data is stationary is often violated in settings like deep reinforcement learning, leading to unstable and brittle learning algorithms. Plasticity loss is attributed to multiple independent mechanisms, and while addressing a single mechanism is insufficient, combining interventions on multiple mechanisms can significantly improve robustness. The study identifies that plasticity loss can be decomposed into several mechanisms, including preactivation distribution shift, regression target magnitude, and parameter growth. It shows that layer normalization and weight decay are effective in maintaining plasticity across various synthetic and real-world nonstationary tasks, including reinforcement learning in the Arcade Learning Environment. The paper also explores the empirical neural tangent kernel (NTK) as a diagnostic tool for plasticity loss, revealing that networks with plasticity loss exhibit similar pathologies in their NTKs. This suggests that the NTK can be used to predict and diagnose training difficulties. The study proposes a "Swiss cheese model" of mitigation strategies, where interventions targeting different mechanisms can be combined to achieve additive benefits. It demonstrates that combining layer normalization with L2 regularization is highly effective in maintaining plasticity in both synthetic and natural distribution shift scenarios. The paper concludes that while no single network property can explain all instances of plasticity loss, several independent mechanisms are responsible for a large fraction of observed cases. By identifying and combining effective interventions for each mechanism, the combinatorial complexity of mitigation strategies can be significantly reduced. This approach offers promising directions for future research in stabilizing optimization in nonstationary learning problems.