Learning by Reconstruction Produces Uninformative Features For Perception

Learning by Reconstruction Produces Uninformative Features For Perception

Feb 2024 | Randall Balestriero, Yann LeCun
Learning by reconstruction often produces features that are uninformative for perception tasks. This paper investigates the misalignment between reconstruction-based learning and learning for perception. It shows that reconstruction focuses on a subspace explaining most of the pixel variance, which is uninformative for perception tasks. In contrast, features useful for perception are learned later, requiring longer training times. The study also demonstrates that while some noise strategies like masking are beneficial, others like additive Gaussian noise are not. The paper proves that masking can improve alignment between reconstruction and perception tasks, while additive noise does not. It also shows that the effectiveness of noise strategies varies with the dataset and the mask's shape and ratio. The findings suggest that careful design of denoising tasks can help align reconstruction with perception tasks, improving performance. The study highlights the need for additional guidance in reconstruction-based learning to focus on features useful for perception. The results indicate that while reconstruction-based methods can produce good reconstructions, they often require fine-tuning to achieve competitive performance on perception tasks. The paper concludes that denoising strategies, particularly masking, can help align reconstruction with perception tasks, improving performance without significantly affecting reconstruction quality.Learning by reconstruction often produces features that are uninformative for perception tasks. This paper investigates the misalignment between reconstruction-based learning and learning for perception. It shows that reconstruction focuses on a subspace explaining most of the pixel variance, which is uninformative for perception tasks. In contrast, features useful for perception are learned later, requiring longer training times. The study also demonstrates that while some noise strategies like masking are beneficial, others like additive Gaussian noise are not. The paper proves that masking can improve alignment between reconstruction and perception tasks, while additive noise does not. It also shows that the effectiveness of noise strategies varies with the dataset and the mask's shape and ratio. The findings suggest that careful design of denoising tasks can help align reconstruction with perception tasks, improving performance. The study highlights the need for additional guidance in reconstruction-based learning to focus on features useful for perception. The results indicate that while reconstruction-based methods can produce good reconstructions, they often require fine-tuning to achieve competitive performance on perception tasks. The paper concludes that denoising strategies, particularly masking, can help align reconstruction with perception tasks, improving performance without significantly affecting reconstruction quality.
Reach us at info@study.space
[slides and audio] Learning by Reconstruction Produces Uninformative Features For Perception