FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

2020 | Kihyuk Sohn*, David Berthelot*, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel
FixMatch is a simplified semi-supervised learning (SSL) algorithm that combines pseudo-labeling and consistency regularization. It generates pseudo-labels using weakly-augmented unlabeled images and retains only high-confidence predictions. The model is then trained to predict these pseudo-labels on strongly-augmented versions of the same images. Despite its simplicity, FixMatch achieves state-of-the-art performance on various SSL benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labeled examples and 88.61% accuracy with only 4 labels per class. The paper includes an extensive ablation study to identify the most important factors contributing to its success, such as regularization, optimization, and hyperparameters. FixMatch is also shown to be effective in extremely low-label regimes and can be extended with techniques like Augmentation Anchoring and Distribution Alignment.FixMatch is a simplified semi-supervised learning (SSL) algorithm that combines pseudo-labeling and consistency regularization. It generates pseudo-labels using weakly-augmented unlabeled images and retains only high-confidence predictions. The model is then trained to predict these pseudo-labels on strongly-augmented versions of the same images. Despite its simplicity, FixMatch achieves state-of-the-art performance on various SSL benchmarks, including 94.93% accuracy on CIFAR-10 with 250 labeled examples and 88.61% accuracy with only 4 labels per class. The paper includes an extensive ablation study to identify the most important factors contributing to its success, such as regularization, optimization, and hyperparameters. FixMatch is also shown to be effective in extremely low-label regimes and can be extended with techniques like Augmentation Anchoring and Distribution Alignment.
Reach us at info@study.space