Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data

Mamba-ND: Selective State Space Modeling for Multi-Dimensional Data

13 Jul 2024 | Shufan Li, Harkanwar Singh, Aditya Grover
Mamba-ND is a state space model (SSM) architecture designed for multi-dimensional data, extending the Mamba model to handle data beyond text sequences. The Mamba model achieves linear complexity in sequence length, unlike Transformers which have quadratic complexity. Mamba-ND introduces a novel approach to process multi-dimensional data by alternating sequence orderings across layers, enabling efficient modeling of 2D and 3D data. It outperforms existing models like Transformers, S4ND, and others on tasks such as image classification, action recognition, weather forecasting, and 3D segmentation, with significantly fewer parameters. The design leverages row-major orderings and alternates between different dimensions to maintain linear complexity. Mamba-ND is evaluated on multiple benchmarks, including ImageNet-1K, HMDB-51, UCF-101, ERA5, and BTCV, demonstrating competitive performance with state-of-the-art models. The architecture is implemented in Python and available on GitHub. The key contributions include the extension of Mamba to multi-dimensional data, achieving strong performance with fewer parameters, and conducting extensive ablation studies showing that simple alternating orderings are effective. The model is efficient, scalable, and suitable for various applications involving multi-dimensional data.Mamba-ND is a state space model (SSM) architecture designed for multi-dimensional data, extending the Mamba model to handle data beyond text sequences. The Mamba model achieves linear complexity in sequence length, unlike Transformers which have quadratic complexity. Mamba-ND introduces a novel approach to process multi-dimensional data by alternating sequence orderings across layers, enabling efficient modeling of 2D and 3D data. It outperforms existing models like Transformers, S4ND, and others on tasks such as image classification, action recognition, weather forecasting, and 3D segmentation, with significantly fewer parameters. The design leverages row-major orderings and alternates between different dimensions to maintain linear complexity. Mamba-ND is evaluated on multiple benchmarks, including ImageNet-1K, HMDB-51, UCF-101, ERA5, and BTCV, demonstrating competitive performance with state-of-the-art models. The architecture is implemented in Python and available on GitHub. The key contributions include the extension of Mamba to multi-dimensional data, achieving strong performance with fewer parameters, and conducting extensive ablation studies showing that simple alternating orderings are effective. The model is efficient, scalable, and suitable for various applications involving multi-dimensional data.
Reach us at info@study.space
Understanding Mamba-ND%3A Selective State Space Modeling for Multi-Dimensional Data