FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

FixMatch: Simplifying Semi-Supervised Learning with Consistency and Confidence

2020 | Kihyuk Sohn*, David Berthelot*, Chun-Liang Li, Zizhao Zhang, Nicholas Carlini, Ekin D. Cubuk, Alex Kurakin, Han Zhang, Colin Raffel
FixMatch is a simplified semi-supervised learning (SSL) algorithm that achieves state-of-the-art performance. It combines consistency regularization and pseudo-labeling to improve model performance using unlabeled data. The algorithm first generates pseudo-labels for weakly-augmented unlabeled images, retaining only those with high confidence. It then trains the model to predict these pseudo-labels on strongly-augmented versions of the same images. FixMatch is simple, requiring fewer hyperparameters and achieving high accuracy on benchmarks like CIFAR-10 with as few as 4 labels per class. It outperforms existing SSL methods, including ReMixMatch and UDA, and is effective in low-label scenarios. The algorithm uses strong augmentations like RandAugment and CTAugment, and includes an ablation study to identify key factors contributing to its success. FixMatch is also effective on larger datasets like ImageNet, achieving better performance than UDA. The simplicity of FixMatch allows for thorough analysis of its components, highlighting the importance of weight decay and optimizer choice. Overall, FixMatch provides a practical and effective approach to semi-supervised learning, particularly in scenarios with limited labeled data.FixMatch is a simplified semi-supervised learning (SSL) algorithm that achieves state-of-the-art performance. It combines consistency regularization and pseudo-labeling to improve model performance using unlabeled data. The algorithm first generates pseudo-labels for weakly-augmented unlabeled images, retaining only those with high confidence. It then trains the model to predict these pseudo-labels on strongly-augmented versions of the same images. FixMatch is simple, requiring fewer hyperparameters and achieving high accuracy on benchmarks like CIFAR-10 with as few as 4 labels per class. It outperforms existing SSL methods, including ReMixMatch and UDA, and is effective in low-label scenarios. The algorithm uses strong augmentations like RandAugment and CTAugment, and includes an ablation study to identify key factors contributing to its success. FixMatch is also effective on larger datasets like ImageNet, achieving better performance than UDA. The simplicity of FixMatch allows for thorough analysis of its components, highlighting the importance of weight decay and optimizer choice. Overall, FixMatch provides a practical and effective approach to semi-supervised learning, particularly in scenarios with limited labeled data.
Reach us at info@study.space