29 Nov 2017 | Terrance DeVries and Graham W. Taylor
This paper introduces Cutout, a simple and effective regularization technique for convolutional neural networks (CNNs). Cutout randomly masks out square regions of input images during training, encouraging the network to learn more robust and generalizable features. Unlike dropout, which randomly deactivates neurons, Cutout masks out contiguous sections of the input, forcing the network to rely on the full context of the image rather than specific visual features. This approach is computationally efficient and can be combined with existing data augmentation and regularization techniques to further improve model performance.
Cutout was evaluated on several popular image recognition datasets, including CIFAR-10, CIFAR-100, SVHN, and STL-10. On CIFAR-10, Cutout reduced test error by 0.4-2.0 percentage points, achieving a new state-of-the-art result of 2.56% test error. On CIFAR-100, it reduced test error by 0.4-2.0 percentage points, achieving a new state-of-the-art result of 15.20% test error. On SVHN, Cutout reduced test error by 0.3 percentage points, achieving a new state-of-the-art result of 1.30% test error. On STL-10, Cutout reduced test error by 2.7 percentage points in the no data-augmentation case and 1.5 percentage points when using data augmentation.
Cutout is particularly effective in scenarios where the model needs to generalize to occluded or partially visible inputs. It encourages the network to learn more robust and generalizable features by forcing it to rely on the full context of the image rather than specific visual features. Cutout is also computationally efficient and can be applied during data loading in parallel with the main training task, making it effectively computationally free.
The technique was found to be complementary to existing forms of data augmentation and regularization. Applied to modern architectures such as wide residual networks and shake-shake regularization models, Cutout achieves state-of-the-art performance on the CIFAR-10, CIFAR-100, and SVHN vision benchmarks. The results suggest that Cutout is a valuable addition to the toolkit of regularization techniques for CNNs.This paper introduces Cutout, a simple and effective regularization technique for convolutional neural networks (CNNs). Cutout randomly masks out square regions of input images during training, encouraging the network to learn more robust and generalizable features. Unlike dropout, which randomly deactivates neurons, Cutout masks out contiguous sections of the input, forcing the network to rely on the full context of the image rather than specific visual features. This approach is computationally efficient and can be combined with existing data augmentation and regularization techniques to further improve model performance.
Cutout was evaluated on several popular image recognition datasets, including CIFAR-10, CIFAR-100, SVHN, and STL-10. On CIFAR-10, Cutout reduced test error by 0.4-2.0 percentage points, achieving a new state-of-the-art result of 2.56% test error. On CIFAR-100, it reduced test error by 0.4-2.0 percentage points, achieving a new state-of-the-art result of 15.20% test error. On SVHN, Cutout reduced test error by 0.3 percentage points, achieving a new state-of-the-art result of 1.30% test error. On STL-10, Cutout reduced test error by 2.7 percentage points in the no data-augmentation case and 1.5 percentage points when using data augmentation.
Cutout is particularly effective in scenarios where the model needs to generalize to occluded or partially visible inputs. It encourages the network to learn more robust and generalizable features by forcing it to rely on the full context of the image rather than specific visual features. Cutout is also computationally efficient and can be applied during data loading in parallel with the main training task, making it effectively computationally free.
The technique was found to be complementary to existing forms of data augmentation and regularization. Applied to modern architectures such as wide residual networks and shake-shake regularization models, Cutout achieves state-of-the-art performance on the CIFAR-10, CIFAR-100, and SVHN vision benchmarks. The results suggest that Cutout is a valuable addition to the toolkit of regularization techniques for CNNs.