This paper investigates the spectral bias of deep neural networks, highlighting their tendency to learn low-frequency functions during training. Using Fourier analysis, the authors show that neural networks prioritize learning functions that vary globally without local fluctuations, a phenomenon referred to as spectral bias. This bias is evident in both the learning process and the parameterization of the model, as lower frequency components are more robust to random parameter perturbations. The study also explores how the shape of the data manifold influences the learning of higher frequencies, finding that complex manifolds facilitate the learning of higher frequencies. The analysis is conducted on ReLU networks, which are continuous piecewise-linear functions, and the results are supported by both empirical and theoretical evidence. The findings suggest that neural networks exhibit a learning bias towards simple, low-frequency functions, which aligns with their ability to generalize across data samples. The study also demonstrates that the geometry of the data manifold plays a crucial role in the expressivity of neural networks, with complex manifolds enabling the learning of higher frequencies. The results are validated through experiments on synthetic and real data, showing that lower frequencies are learned first and that the network's performance is more robust to parameter perturbations for lower frequencies. The study concludes that the spectral bias of neural networks is a fundamental property that influences their learning and generalization capabilities.This paper investigates the spectral bias of deep neural networks, highlighting their tendency to learn low-frequency functions during training. Using Fourier analysis, the authors show that neural networks prioritize learning functions that vary globally without local fluctuations, a phenomenon referred to as spectral bias. This bias is evident in both the learning process and the parameterization of the model, as lower frequency components are more robust to random parameter perturbations. The study also explores how the shape of the data manifold influences the learning of higher frequencies, finding that complex manifolds facilitate the learning of higher frequencies. The analysis is conducted on ReLU networks, which are continuous piecewise-linear functions, and the results are supported by both empirical and theoretical evidence. The findings suggest that neural networks exhibit a learning bias towards simple, low-frequency functions, which aligns with their ability to generalize across data samples. The study also demonstrates that the geometry of the data manifold plays a crucial role in the expressivity of neural networks, with complex manifolds enabling the learning of higher frequencies. The results are validated through experiments on synthetic and real data, showing that lower frequencies are learned first and that the network's performance is more robust to parameter perturbations for lower frequencies. The study concludes that the spectral bias of neural networks is a fundamental property that influences their learning and generalization capabilities.