1 Jul 2020 | Olivier J. Hénaff, Aravind Srinivas, Jeffrey De Fauw, Ali Razavi, Carl Doersch, S. M. Ali Eslami, Aaron van den Oord
The paper "Data-Efficient Image Recognition with Contrastive Predictive Coding" by Olivier J. Hénaff et al. explores the hypothesis that data-efficient image recognition can be achieved by learning representations that make natural signal variability more predictable. The authors revisit and improve the Contrastive Predictive Coding (CPC) objective, an unsupervised method for learning such representations. The new implementation of CPC produces features that achieve state-of-the-art linear classification accuracy on the ImageNet dataset. When used as input for non-linear classification with deep neural networks, these features enable the use of 2–5× fewer labels than classifiers trained directly on image pixels. Additionally, the unsupervised representation significantly improves transfer learning to object detection on the PASCAL VOC dataset, surpassing fully supervised pre-trained ImageNet classifiers. The paper contributes to the field of data-efficient image recognition by demonstrating the effectiveness of CPC in both linear classification and semi-supervised learning, and by showing its superior performance in transfer learning compared to supervised methods.The paper "Data-Efficient Image Recognition with Contrastive Predictive Coding" by Olivier J. Hénaff et al. explores the hypothesis that data-efficient image recognition can be achieved by learning representations that make natural signal variability more predictable. The authors revisit and improve the Contrastive Predictive Coding (CPC) objective, an unsupervised method for learning such representations. The new implementation of CPC produces features that achieve state-of-the-art linear classification accuracy on the ImageNet dataset. When used as input for non-linear classification with deep neural networks, these features enable the use of 2–5× fewer labels than classifiers trained directly on image pixels. Additionally, the unsupervised representation significantly improves transfer learning to object detection on the PASCAL VOC dataset, surpassing fully supervised pre-trained ImageNet classifiers. The paper contributes to the field of data-efficient image recognition by demonstrating the effectiveness of CPC in both linear classification and semi-supervised learning, and by showing its superior performance in transfer learning compared to supervised methods.