This paper introduces a novel regularization method for deep convolutional neural networks called stochastic pooling. Instead of using deterministic pooling operations like average or max pooling, the method randomly selects the activation within each pooling region according to a multinomial distribution based on the activities within the region. This approach is hyper-parameter free and can be combined with other regularization techniques such as dropout and data augmentation. The method achieves state-of-the-art performance on four image datasets, relative to other approaches that do not utilize data augmentation.
The key idea is to make the pooling process stochastic, which allows the network to learn more robust features by considering multiple possible activations within each pooling region. This approach prevents overfitting and improves performance on recognition tasks. Stochastic pooling can be viewed as equivalent to standard max pooling but with many copies of an input image, each having small local deformations. This is similar to explicit elastic deformations of the input images, which delivers excellent performance on the MNIST dataset.
The paper evaluates the method on several image datasets including CIFAR-10, CIFAR-100, MNIST, and SVHN. The results show that stochastic pooling outperforms both average and max pooling in terms of test accuracy. The method is also effective in preventing overfitting, even when training on smaller datasets. Additionally, the method is computationally efficient and does not require hyper-parameters to tune.
The paper also discusses the importance of model averaging, where the probabilities from stochastic pooling are used to average over multiple models. This approach is shown to be effective in improving performance without requiring the instantiation of multiple models. The method is also compared to other regularization techniques and shown to be effective in preventing overfitting and improving performance on various tasks.This paper introduces a novel regularization method for deep convolutional neural networks called stochastic pooling. Instead of using deterministic pooling operations like average or max pooling, the method randomly selects the activation within each pooling region according to a multinomial distribution based on the activities within the region. This approach is hyper-parameter free and can be combined with other regularization techniques such as dropout and data augmentation. The method achieves state-of-the-art performance on four image datasets, relative to other approaches that do not utilize data augmentation.
The key idea is to make the pooling process stochastic, which allows the network to learn more robust features by considering multiple possible activations within each pooling region. This approach prevents overfitting and improves performance on recognition tasks. Stochastic pooling can be viewed as equivalent to standard max pooling but with many copies of an input image, each having small local deformations. This is similar to explicit elastic deformations of the input images, which delivers excellent performance on the MNIST dataset.
The paper evaluates the method on several image datasets including CIFAR-10, CIFAR-100, MNIST, and SVHN. The results show that stochastic pooling outperforms both average and max pooling in terms of test accuracy. The method is also effective in preventing overfitting, even when training on smaller datasets. Additionally, the method is computationally efficient and does not require hyper-parameters to tune.
The paper also discusses the importance of model averaging, where the probabilities from stochastic pooling are used to average over multiple models. This approach is shown to be effective in improving performance without requiring the instantiation of multiple models. The method is also compared to other regularization techniques and shown to be effective in preventing overfitting and improving performance on various tasks.