Improved Baselines with Momentum Contrastive Learning

Improved Baselines with Momentum Contrastive Learning

9 Mar 2020 | Xinlei Chen Haoqi Fan Ross Girshick Kaiming He
This paper explores the effectiveness of two design improvements from SimCLR within the Momentum Contrast (MoCo) framework, aiming to establish stronger and more feasible baselines for unsupervised learning. The improvements include using an MLP projection head and stronger data augmentation. These modifications are orthogonal to the existing MoCo framework and lead to better image classification and object detection transfer learning results. The authors demonstrate that their "MoCo v2" baselines can achieve better performance than SimCLR without requiring large training batches, making state-of-the-art unsupervised learning more accessible. The paper also reports improved accuracy on ImageNet linear classification and VOC object detection tasks, and provides detailed experimental results and comparisons with SimCLR. The code for these improvements will be made public to facilitate future research.This paper explores the effectiveness of two design improvements from SimCLR within the Momentum Contrast (MoCo) framework, aiming to establish stronger and more feasible baselines for unsupervised learning. The improvements include using an MLP projection head and stronger data augmentation. These modifications are orthogonal to the existing MoCo framework and lead to better image classification and object detection transfer learning results. The authors demonstrate that their "MoCo v2" baselines can achieve better performance than SimCLR without requiring large training batches, making state-of-the-art unsupervised learning more accessible. The paper also reports improved accuracy on ImageNet linear classification and VOC object detection tasks, and provides detailed experimental results and comparisons with SimCLR. The code for these improvements will be made public to facilitate future research.
Reach us at info@study.space
[slides] Improved Baselines with Momentum Contrastive Learning | StudySpace