The paper introduces Pretext-Invariant Representation Learning (PIRL), a method for learning image representations that are invariant to image transformations, rather than covariant as in traditional pretext tasks. PIRL aims to construct representations that remain similar to the transformed versions of the same image but differ from other images, thereby preserving semantic information. The authors use the Jigsaw pretext task, which involves solving jigsaw puzzles, to demonstrate the effectiveness of PIRL. They find that PIRL significantly improves the semantic quality of learned image representations, outperforming both self-supervised and supervised pre-training methods in various benchmarks, including object detection and image classification. PIRL sets a new state-of-the-art in self-supervised learning from images, showing that learning invariant representations can lead to better performance in downstream tasks. The paper also includes experiments to analyze the properties of PIRL representations, the impact of hyperparameters, and the generalization of PIRL to other pretext tasks.The paper introduces Pretext-Invariant Representation Learning (PIRL), a method for learning image representations that are invariant to image transformations, rather than covariant as in traditional pretext tasks. PIRL aims to construct representations that remain similar to the transformed versions of the same image but differ from other images, thereby preserving semantic information. The authors use the Jigsaw pretext task, which involves solving jigsaw puzzles, to demonstrate the effectiveness of PIRL. They find that PIRL significantly improves the semantic quality of learned image representations, outperforming both self-supervised and supervised pre-training methods in various benchmarks, including object detection and image classification. PIRL sets a new state-of-the-art in self-supervised learning from images, showing that learning invariant representations can lead to better performance in downstream tasks. The paper also includes experiments to analyze the properties of PIRL representations, the impact of hyperparameters, and the generalization of PIRL to other pretext tasks.