This paper introduces a novel network design paradigm that focuses on designing network design spaces rather than individual network instances. The authors aim to advance the understanding of network design and discover generalizable design principles. They start with an initial, unconstrained design space called *AnyNet* and progressively refine it to a low-dimensional, regular design space called *RegNet*. The core insight of the RegNet parametrization is that the widths and depths of good networks can be explained by a quantized linear function. The RegNet design space is found to be more interpretable and efficient, with simpler models that perform well across a wide range of flop regimes. The authors compare RegNet models to popular networks like EfficientNet and find that they outperform them while being up to 5× faster on GPUs. The paper also provides insights into common deep network design choices, such as the depth and width of networks, and discusses the generalization of the RegNet design space to different settings.This paper introduces a novel network design paradigm that focuses on designing network design spaces rather than individual network instances. The authors aim to advance the understanding of network design and discover generalizable design principles. They start with an initial, unconstrained design space called *AnyNet* and progressively refine it to a low-dimensional, regular design space called *RegNet*. The core insight of the RegNet parametrization is that the widths and depths of good networks can be explained by a quantized linear function. The RegNet design space is found to be more interpretable and efficient, with simpler models that perform well across a wide range of flop regimes. The authors compare RegNet models to popular networks like EfficientNet and find that they outperform them while being up to 5× faster on GPUs. The paper also provides insights into common deep network design choices, such as the depth and width of networks, and discusses the generalization of the RegNet design space to different settings.