16 Apr 2024 | Oren Kraus, Kian Kenyon-Dean, Saber Saberian, Maryam Fallah, Peter McLean, Jess Leung, Vasudev Sharma, Ayla Khan, Jia Balakrishnan, Safiye Celik, Dominique Beaini, Maciej Sypetkowski, Chi Vicky Cheng, Kristen Morse, Maureen Makes, Ben Mabey, Berton Earnshaw
This work presents Masked Autoencoders (MAEs) for Microscopy, demonstrating their effectiveness in learning cellular biology from large-scale microscopy datasets. The study evaluates the scalability of weakly supervised classifiers and MAEs, showing that ViT-based MAEs outperform weakly supervised classifiers in tasks such as recalling known biological relationships, achieving up to 11.5% relative improvement. A new channel-agnostic MAE architecture (CA-MAE) is introduced, enabling input of images with varying numbers and orders of channels. CA-MAEs generalize well to microscopy datasets with different channel configurations, as demonstrated on the JUMP-CP dataset generated under different experimental conditions. The findings suggest that scaling self-supervised learning on microscopy data can lead to powerful foundation models of cellular biology, with potential applications in drug discovery and beyond. The study also introduces a Fourier domain reconstruction loss to stabilize training of large ViT backbones and shows that CA-MAEs outperform standard MAEs in performance and generalization. Results indicate that MAEs are scalable learners of cellular biology, outperforming previous state-of-the-art methods in inferring known biological relationships. The study compares MAEs with other models, including weakly supervised learning (WSL) and pre-trained ImageNet models, showing that MAEs achieve better performance, especially when trained on larger datasets. The study also demonstrates that MAEs can predict morphological features and perform well in tasks such as perturbation retrieval and sibling retrieval. The results highlight the potential of MAEs in advancing biological research through scalable, self-supervised learning on microscopy data.This work presents Masked Autoencoders (MAEs) for Microscopy, demonstrating their effectiveness in learning cellular biology from large-scale microscopy datasets. The study evaluates the scalability of weakly supervised classifiers and MAEs, showing that ViT-based MAEs outperform weakly supervised classifiers in tasks such as recalling known biological relationships, achieving up to 11.5% relative improvement. A new channel-agnostic MAE architecture (CA-MAE) is introduced, enabling input of images with varying numbers and orders of channels. CA-MAEs generalize well to microscopy datasets with different channel configurations, as demonstrated on the JUMP-CP dataset generated under different experimental conditions. The findings suggest that scaling self-supervised learning on microscopy data can lead to powerful foundation models of cellular biology, with potential applications in drug discovery and beyond. The study also introduces a Fourier domain reconstruction loss to stabilize training of large ViT backbones and shows that CA-MAEs outperform standard MAEs in performance and generalization. Results indicate that MAEs are scalable learners of cellular biology, outperforming previous state-of-the-art methods in inferring known biological relationships. The study compares MAEs with other models, including weakly supervised learning (WSL) and pre-trained ImageNet models, showing that MAEs achieve better performance, especially when trained on larger datasets. The study also demonstrates that MAEs can predict morphological features and perform well in tasks such as perturbation retrieval and sibling retrieval. The results highlight the potential of MAEs in advancing biological research through scalable, self-supervised learning on microscopy data.