nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Model

nnMamba: 3D Biomedical Image Segmentation, Classification and Landmark Detection with State Space Model

10 Mar 2024 | Haifan Gong, Luoyao Kang, Yitao Wang, Xiang Wan, Haofeng Li
nnMamba is a novel architecture that integrates the strengths of Convolutional Neural Networks (CNNs) and the advanced long-range modeling capabilities of State Space Sequence Models (SSMs), designed for 3D biomedical image segmentation, classification, and landmark detection. The paper introduces the Mamba-In-Convolution with Channel-Spatial Siamese learning (MICCSS) block to model the long-range relationship of voxels. For dense prediction and classification tasks, the paper also proposes channel-scaling and channel-sequential learning methods. Extensive experiments on six datasets demonstrate that nnMamba outperforms state-of-the-art methods in 3D image segmentation, classification, and landmark detection. nnMamba combines the local representation ability of CNNs with the efficient global context processing of SSMs, setting a new standard for long-range dependency modeling in medical image analysis. The MICCSS block is designed to enable long-range relationship modeling at both channel and spatial levels. The MIC block integrates SSM with Convolutional layers, while the CSS module leverages SSM for channel and spatial feature interaction. For segmentation and landmark detection, the paper implements a UNet architecture to capture long-range dependencies between the encoder and decoder. For classification tasks, the paper proposes a residual encoder-centric architecture, incorporating the Mamba layer early in the network to capture long-range dependencies. The paper evaluates nnMamba on the BraTS 2023 GIL track, the AMOS2022 dataset, the ADNI dataset, and a private fetal cerebellum landmark detection dataset. The results show that nnMamba achieves superior performance in all tasks, with significant improvements in segmentation, classification, and landmark detection. The ablation study demonstrates that the integration of MIC and MICCSS leads to consistent improvements in segmentation metrics, landmark detection, and classification tasks. The visualization results show that nnMamba can better capture the discontinuous regions of the tumor. The paper concludes that nnMamba is a robust solution for 3D medical image analysis, offering both the local representation ability of CNNs and the efficient global context processing of SSMs.nnMamba is a novel architecture that integrates the strengths of Convolutional Neural Networks (CNNs) and the advanced long-range modeling capabilities of State Space Sequence Models (SSMs), designed for 3D biomedical image segmentation, classification, and landmark detection. The paper introduces the Mamba-In-Convolution with Channel-Spatial Siamese learning (MICCSS) block to model the long-range relationship of voxels. For dense prediction and classification tasks, the paper also proposes channel-scaling and channel-sequential learning methods. Extensive experiments on six datasets demonstrate that nnMamba outperforms state-of-the-art methods in 3D image segmentation, classification, and landmark detection. nnMamba combines the local representation ability of CNNs with the efficient global context processing of SSMs, setting a new standard for long-range dependency modeling in medical image analysis. The MICCSS block is designed to enable long-range relationship modeling at both channel and spatial levels. The MIC block integrates SSM with Convolutional layers, while the CSS module leverages SSM for channel and spatial feature interaction. For segmentation and landmark detection, the paper implements a UNet architecture to capture long-range dependencies between the encoder and decoder. For classification tasks, the paper proposes a residual encoder-centric architecture, incorporating the Mamba layer early in the network to capture long-range dependencies. The paper evaluates nnMamba on the BraTS 2023 GIL track, the AMOS2022 dataset, the ADNI dataset, and a private fetal cerebellum landmark detection dataset. The results show that nnMamba achieves superior performance in all tasks, with significant improvements in segmentation, classification, and landmark detection. The ablation study demonstrates that the integration of MIC and MICCSS leads to consistent improvements in segmentation metrics, landmark detection, and classification tasks. The visualization results show that nnMamba can better capture the discontinuous regions of the tumor. The paper concludes that nnMamba is a robust solution for 3D medical image analysis, offering both the local representation ability of CNNs and the efficient global context processing of SSMs.
Reach us at info@study.space
[slides] nnMamba%3A 3D Biomedical Image Segmentation%2C Classification and Landmark Detection with State Space Model | StudySpace