The paper "Do Better ImageNet Models Transfer Better?" by Simon Kornblith, Jonathon Shlens, and Quoc V. Le from Google Brain explores the relationship between the performance of models on the ImageNet dataset and their transferability to other image classification tasks. The authors investigate 16 modern convolutional neural networks (CNNs) on 12 image classification datasets in three experimental settings: as fixed feature extractors, fine-tuned from ImageNet initialization, and trained from random initialization. They find a strong correlation between ImageNet accuracy and transfer accuracy (r = 0.99 for fixed features and r = 0.96 for fine-tuning). However, they also discover that common regularization techniques used in ImageNet training, such as label smoothing and dropout, can significantly improve ImageNet accuracy but harm the performance of transfer learning based on penultimate layer features. Additionally, pretraining on ImageNet provides minimal benefits on two small fine-grained image classification datasets, indicating that learned features from ImageNet do not generalize well to fine-grained tasks. The results suggest that while ImageNet architectures generalize well across datasets, the learned features from ImageNet are less general than previously thought. The paper also discusses the implications of these findings for the field of transfer learning and the design of more effective models for transfer tasks.The paper "Do Better ImageNet Models Transfer Better?" by Simon Kornblith, Jonathon Shlens, and Quoc V. Le from Google Brain explores the relationship between the performance of models on the ImageNet dataset and their transferability to other image classification tasks. The authors investigate 16 modern convolutional neural networks (CNNs) on 12 image classification datasets in three experimental settings: as fixed feature extractors, fine-tuned from ImageNet initialization, and trained from random initialization. They find a strong correlation between ImageNet accuracy and transfer accuracy (r = 0.99 for fixed features and r = 0.96 for fine-tuning). However, they also discover that common regularization techniques used in ImageNet training, such as label smoothing and dropout, can significantly improve ImageNet accuracy but harm the performance of transfer learning based on penultimate layer features. Additionally, pretraining on ImageNet provides minimal benefits on two small fine-grained image classification datasets, indicating that learned features from ImageNet do not generalize well to fine-grained tasks. The results suggest that while ImageNet architectures generalize well across datasets, the learned features from ImageNet are less general than previously thought. The paper also discusses the implications of these findings for the field of transfer learning and the design of more effective models for transfer tasks.