Temporal Ensembling for Semi-Supervised Learning

Temporal Ensembling for Semi-Supervised Learning

15 Mar 2017 | Samuli Laine, Timo Aila
This paper introduces a method called self-ensembling for semi-supervised learning, where only a small portion of the training data is labeled. The method formulates a consensus prediction of unknown labels using the outputs of a network trained on different epochs under varying regularization and input augmentation conditions. This ensemble prediction is used as a target for training, which significantly improves classification accuracy compared to traditional methods. The authors present two implementations of self-ensembling: the Π-model and temporal ensembling. Both methods outperform prior state-of-the-art results in semi-supervised learning benchmarks, reducing classification error rates in SVHN and CIFAR-10 by up to 65% and 43%, respectively. The temporal ensembling method is also faster and more robust to incorrect labels. The paper discusses the relationship between self-ensembling and other semi-supervised learning techniques, such as the transform/stability loss and bootstrapping methods.This paper introduces a method called self-ensembling for semi-supervised learning, where only a small portion of the training data is labeled. The method formulates a consensus prediction of unknown labels using the outputs of a network trained on different epochs under varying regularization and input augmentation conditions. This ensemble prediction is used as a target for training, which significantly improves classification accuracy compared to traditional methods. The authors present two implementations of self-ensembling: the Π-model and temporal ensembling. Both methods outperform prior state-of-the-art results in semi-supervised learning benchmarks, reducing classification error rates in SVHN and CIFAR-10 by up to 65% and 43%, respectively. The temporal ensembling method is also faster and more robust to incorrect labels. The paper discusses the relationship between self-ensembling and other semi-supervised learning techniques, such as the transform/stability loss and bootstrapping methods.
Reach us at info@study.space