25 Jun 2024 | Jiahao Huang, Liutao Yang, Fanwen Wang, Yang Nan, Angelica I. Aviles-Rivero, Carola-Bibiane Schönlieb, Daoqiang Zhang, Guang Yang
MambaMIR is a Mamba-based model designed for joint medical image reconstruction and uncertainty estimation, along with its GAN variant, MambaMIR-GAN. The model leverages the advantages of the original Mamba, including linear complexity, global receptive fields, and dynamic weights, to efficiently process long sequences for medical image reconstruction. An arbitrary-mask mechanism is introduced to adapt Mamba to image reconstruction tasks, providing randomness for Monte Carlo-based uncertainty estimation. Experiments on various medical image reconstruction tasks, including fast MRI and sparse-view CT (SVCT), demonstrate that MambaMIR and MambaMIR-GAN achieve comparable or superior reconstruction results compared to state-of-the-art methods. The estimated uncertainty maps offer insights into the reliability of the reconstruction quality. The model's architecture includes an Input Module, a series of Arbitrary-Masked State Space (AMSS) Block Groups, and an Output Module, with residual connections for effective feature processing. The AMSS Block integrates four key modules: Scan Expanding, Arbitrary-Masked, S6, and Scan Merging. The model is trained using a combination of image and transform domain losses, along with perceptual losses in the latent space of a pre-trained VGG. MambaMIR-GAN incorporates adversarial training, enhancing perceptual quality. The model achieves high PSNR, SSIM, and LPIPS scores on various datasets, demonstrating its effectiveness in medical image reconstruction. The uncertainty maps generated by MambaMIR provide visual confidence indicators for reconstructed images, highlighting areas of potential uncertainty. The model shows promising results in both MRI and SVCT reconstruction, with MambaMIR-GAN outperforming in perceptual quality and uncertainty estimation. The study highlights the potential of MambaMIR and MambaMIR-GAN in advancing medical image reconstruction with improved efficiency and reliability.MambaMIR is a Mamba-based model designed for joint medical image reconstruction and uncertainty estimation, along with its GAN variant, MambaMIR-GAN. The model leverages the advantages of the original Mamba, including linear complexity, global receptive fields, and dynamic weights, to efficiently process long sequences for medical image reconstruction. An arbitrary-mask mechanism is introduced to adapt Mamba to image reconstruction tasks, providing randomness for Monte Carlo-based uncertainty estimation. Experiments on various medical image reconstruction tasks, including fast MRI and sparse-view CT (SVCT), demonstrate that MambaMIR and MambaMIR-GAN achieve comparable or superior reconstruction results compared to state-of-the-art methods. The estimated uncertainty maps offer insights into the reliability of the reconstruction quality. The model's architecture includes an Input Module, a series of Arbitrary-Masked State Space (AMSS) Block Groups, and an Output Module, with residual connections for effective feature processing. The AMSS Block integrates four key modules: Scan Expanding, Arbitrary-Masked, S6, and Scan Merging. The model is trained using a combination of image and transform domain losses, along with perceptual losses in the latent space of a pre-trained VGG. MambaMIR-GAN incorporates adversarial training, enhancing perceptual quality. The model achieves high PSNR, SSIM, and LPIPS scores on various datasets, demonstrating its effectiveness in medical image reconstruction. The uncertainty maps generated by MambaMIR provide visual confidence indicators for reconstructed images, highlighting areas of potential uncertainty. The model shows promising results in both MRI and SVCT reconstruction, with MambaMIR-GAN outperforming in perceptual quality and uncertainty estimation. The study highlights the potential of MambaMIR and MambaMIR-GAN in advancing medical image reconstruction with improved efficiency and reliability.