Content-Aware Image Restoration: Pushing the Limits of Fluorescence Microscopy

Content-Aware Image Restoration: Pushing the Limits of Fluorescence Microscopy

| Martin Weigert1,2,*, Uwe Schmidt1,2, Tobias Boothe2, Andreas Müller8,9,10, Alexandr Dibrov1,2, Akanksha Jain2, Benjamin Wilhelm1,6, Deborah Schmidt1,2, Coleman Broaddus1,2, Siân Culley4,5, Mauricio Rocha-Martins1,2, Fabián Segovia-Miranda2, Caren Norden2, Ricardo Henriques4,5, Marino Zerial1,2, Michele Solimena2,8,9,10, Jochen Rink2, Pavel Tomancak2, Loic Royer1,2,7,*, Florian Jug1,2,*,† & Eugene W. Myers1,2,3,†
Content-Aware Image Restoration (CARE) extends the range of biological phenomena observable by fluorescence microscopy. This study demonstrates how deep learning-based image restoration can enhance image quality, enabling the recovery of important biological information that would otherwise be inaccessible. CARE networks are trained to restore images with significantly reduced photon exposure, achieve near-isotropic resolution, and resolve sub-diffraction structures at higher frame rates compared to state-of-the-art methods. The developed methods are available as open-source software in Python, Fiji, and KNIME. Fluorescence microscopy is limited by optical and fluorophore constraints, requiring trade-offs between imaging speed, resolution, light exposure, and depth. CARE networks overcome these limitations by leveraging deep learning to restore images with low signal-to-noise ratios (SNR), achieving high-quality results even with 60-fold fewer photons. They also enable near-isotropic resolution with up to 10-fold under-sampling along the axial direction and allow the resolution of sub-diffraction structures at 20 times higher frame rates than existing methods. The study presents results across multiple imaging scenarios, showing that CARE networks produce results previously unobtainable. For example, in the case of flatworms, CARE enables imaging without unwanted muscle contractions, while for beetle embryos, it allows longer and faster imaging. In Drosophila wing imaging, CARE enables increased temporal resolution and isotropic restoration of 3D volumes. For zebrafish retinas and mouse livers, CARE improves axial resolution and enables more accurate segmentation and tracking of cellular structures. CARE networks also demonstrate the ability to resolve sub-diffraction structures at high frame rates, surpassing traditional methods like deconvolution and super-resolution radial fluctuations (SRRF). They achieve this by using synthetic training data to enhance image resolution, enabling the restoration of structures imperceptible in widefield images. CARE networks also provide reliability metrics, such as per-pixel confidence intervals and ensemble disagreement scores, to assess the accuracy of image restoration. The study highlights the potential of CARE networks to improve biological imaging by enabling higher frame rates, shorter exposures, and lower light intensities while maintaining high resolution. The open-source tools provided make these advancements accessible to the scientific community, with the potential to revolutionize biological image restoration and provide new insights into biological systems.Content-Aware Image Restoration (CARE) extends the range of biological phenomena observable by fluorescence microscopy. This study demonstrates how deep learning-based image restoration can enhance image quality, enabling the recovery of important biological information that would otherwise be inaccessible. CARE networks are trained to restore images with significantly reduced photon exposure, achieve near-isotropic resolution, and resolve sub-diffraction structures at higher frame rates compared to state-of-the-art methods. The developed methods are available as open-source software in Python, Fiji, and KNIME. Fluorescence microscopy is limited by optical and fluorophore constraints, requiring trade-offs between imaging speed, resolution, light exposure, and depth. CARE networks overcome these limitations by leveraging deep learning to restore images with low signal-to-noise ratios (SNR), achieving high-quality results even with 60-fold fewer photons. They also enable near-isotropic resolution with up to 10-fold under-sampling along the axial direction and allow the resolution of sub-diffraction structures at 20 times higher frame rates than existing methods. The study presents results across multiple imaging scenarios, showing that CARE networks produce results previously unobtainable. For example, in the case of flatworms, CARE enables imaging without unwanted muscle contractions, while for beetle embryos, it allows longer and faster imaging. In Drosophila wing imaging, CARE enables increased temporal resolution and isotropic restoration of 3D volumes. For zebrafish retinas and mouse livers, CARE improves axial resolution and enables more accurate segmentation and tracking of cellular structures. CARE networks also demonstrate the ability to resolve sub-diffraction structures at high frame rates, surpassing traditional methods like deconvolution and super-resolution radial fluctuations (SRRF). They achieve this by using synthetic training data to enhance image resolution, enabling the restoration of structures imperceptible in widefield images. CARE networks also provide reliability metrics, such as per-pixel confidence intervals and ensemble disagreement scores, to assess the accuracy of image restoration. The study highlights the potential of CARE networks to improve biological imaging by enabling higher frame rates, shorter exposures, and lower light intensities while maintaining high resolution. The open-source tools provided make these advancements accessible to the scientific community, with the potential to revolutionize biological image restoration and provide new insights into biological systems.
Reach us at info@study.space