3 Jul 2012 | G. E. Hinton*, N. Srivastava, A. Krizhevsky, I. Sutskever and R. R. Salakhutdinov
The paper introduces dropout, a technique to prevent overfitting in neural networks by randomly omitting feature detectors during training. This approach reduces the co-adaptation of feature detectors, allowing each neuron to learn a more general and robust representation. Dropout is applied to various benchmarks, including MNIST, TIMIT, Reuters, CIFAR-10, and ImageNet, showing significant improvements in performance and generalization. The method is particularly effective in deep networks, where it helps control overfitting and allows for larger networks to be trained without early stopping. The paper also discusses the implementation details, hyperparameters, and comparisons with other methods, demonstrating the effectiveness of dropout in improving the robustness and performance of neural networks.The paper introduces dropout, a technique to prevent overfitting in neural networks by randomly omitting feature detectors during training. This approach reduces the co-adaptation of feature detectors, allowing each neuron to learn a more general and robust representation. Dropout is applied to various benchmarks, including MNIST, TIMIT, Reuters, CIFAR-10, and ImageNet, showing significant improvements in performance and generalization. The method is particularly effective in deep networks, where it helps control overfitting and allows for larger networks to be trained without early stopping. The paper also discusses the implementation details, hyperparameters, and comparisons with other methods, demonstrating the effectiveness of dropout in improving the robustness and performance of neural networks.