This paper introduces cyclical learning rates (CLR) as a method for training deep neural networks. Unlike traditional methods that use a fixed or monotonically decreasing learning rate, CLR allows the learning rate to vary cyclically between a minimum and maximum value. This approach eliminates the need for extensive experimentation to find the optimal learning rate and schedule, and often leads to improved classification accuracy with fewer iterations. The paper also describes a simple way to estimate reasonable bounds for the learning rate by increasing the learning rate linearly for a few epochs. The CLR method is demonstrated on several datasets and architectures, including CIFAR-10, CIFAR-100, and ImageNet, using ResNets, Stochastic Depth networks, DenseNets, AlexNet, and GoogLeNet. The results show that CLR achieves near-optimal classification accuracy, often with fewer iterations than traditional methods. The method is computationally efficient and can be combined with adaptive learning rate methods. The paper also discusses how to estimate the minimum and maximum learning rate boundaries using a "LR range test," where the learning rate is increased linearly for a few epochs. The results from this test are used to set the bounds for the CLR policy. The paper concludes that CLR is a practical and effective method for training neural networks, offering significant improvements in performance and reducing the need for manual tuning of learning rates.This paper introduces cyclical learning rates (CLR) as a method for training deep neural networks. Unlike traditional methods that use a fixed or monotonically decreasing learning rate, CLR allows the learning rate to vary cyclically between a minimum and maximum value. This approach eliminates the need for extensive experimentation to find the optimal learning rate and schedule, and often leads to improved classification accuracy with fewer iterations. The paper also describes a simple way to estimate reasonable bounds for the learning rate by increasing the learning rate linearly for a few epochs. The CLR method is demonstrated on several datasets and architectures, including CIFAR-10, CIFAR-100, and ImageNet, using ResNets, Stochastic Depth networks, DenseNets, AlexNet, and GoogLeNet. The results show that CLR achieves near-optimal classification accuracy, often with fewer iterations than traditional methods. The method is computationally efficient and can be combined with adaptive learning rate methods. The paper also discusses how to estimate the minimum and maximum learning rate boundaries using a "LR range test," where the learning rate is increased linearly for a few epochs. The results from this test are used to set the bounds for the CLR policy. The paper concludes that CLR is a practical and effective method for training neural networks, offering significant improvements in performance and reducing the need for manual tuning of learning rates.