This perspective explores the mechanisms underlying generalization in both biological and artificial deep neural networks (DNNs). It introduces two key hypotheses: first, the geometric properties of neural manifolds associated with discrete cognitive entities are powerful order parameters that link neural substrates to generalization capabilities, providing a unified methodology across neuroscience, machine learning, and cognitive science. Second, the theory of learning in wide DNNs, especially in the thermodynamic limit, offers insights into the learning processes that generate desired neural representational geometries and generalization. The paper discusses recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and explores the relation between manifold dimension and radius and generalization capacity. It also delves into the dynamics of learning and its relevance to representational drift in the brain. The authors apply these theories to understand few-shot learning and compare the geometry of DNNs with the primate visual pathway, highlighting differences in manifold dimensionality and structure. Additionally, they discuss the alignment between vision and language representations, suggesting a similar fine-grained, generalizable semantic structure across modalities. The paper concludes by exploring the theory of deep learning, including Langevin learning, kernel renormalization, and feature learning in wide networks, and its implications for representational geometry and generalization.This perspective explores the mechanisms underlying generalization in both biological and artificial deep neural networks (DNNs). It introduces two key hypotheses: first, the geometric properties of neural manifolds associated with discrete cognitive entities are powerful order parameters that link neural substrates to generalization capabilities, providing a unified methodology across neuroscience, machine learning, and cognitive science. Second, the theory of learning in wide DNNs, especially in the thermodynamic limit, offers insights into the learning processes that generate desired neural representational geometries and generalization. The paper discusses recent progress in studying the geometry of neural manifolds, particularly in visual object recognition, and explores the relation between manifold dimension and radius and generalization capacity. It also delves into the dynamics of learning and its relevance to representational drift in the brain. The authors apply these theories to understand few-shot learning and compare the geometry of DNNs with the primate visual pathway, highlighting differences in manifold dimensionality and structure. Additionally, they discuss the alignment between vision and language representations, suggesting a similar fine-grained, generalizable semantic structure across modalities. The paper concludes by exploring the theory of deep learning, including Langevin learning, kernel renormalization, and feature learning in wide networks, and its implications for representational geometry and generalization.