This paper explores the spectral bias of neural networks, particularly ReLU networks, using Fourier analysis. The authors find that deep networks tend to learn low-frequency functions first, a phenomenon they term the *spectral bias*. This bias is reflected in the network's parameterization, where lower frequency components are more robust to random perturbations. The study also investigates the impact of the data manifold's shape on the learnability of high frequencies, finding that complex manifolds can facilitate the learning of higher frequencies. The results suggest that neural networks prioritize simple, smooth functions during training, which aligns with observations of implicit regularization in over-parameterized models. The paper contributes to the understanding of how neural networks approximate functions and provides insights into the interplay between network architecture, data geometry, and learning dynamics.This paper explores the spectral bias of neural networks, particularly ReLU networks, using Fourier analysis. The authors find that deep networks tend to learn low-frequency functions first, a phenomenon they term the *spectral bias*. This bias is reflected in the network's parameterization, where lower frequency components are more robust to random perturbations. The study also investigates the impact of the data manifold's shape on the learnability of high frequencies, finding that complex manifolds can facilitate the learning of higher frequencies. The results suggest that neural networks prioritize simple, smooth functions during training, which aligns with observations of implicit regularization in over-parameterized models. The paper contributes to the understanding of how neural networks approximate functions and provides insights into the interplay between network architecture, data geometry, and learning dynamics.