Mamba is a linear-time sequence modeling architecture that improves upon existing state space models (SSMs) by introducing a selection mechanism that allows the model to selectively propagate or forget information along the sequence length dimension. This mechanism enables the model to focus on relevant information and ignore irrelevant information, making it more effective at modeling discrete modalities such as language. Mamba is designed to be efficient and scalable, with linear scaling in sequence length and fast inference speeds. It achieves state-of-the-art performance across several modalities, including language, audio, and genomics. On language modeling, Mamba-3B outperforms Transformers of the same size and matches Transformers twice its size in both pretraining and downstream evaluation. Mamba is a simplified end-to-end neural network architecture without attention or even MLP blocks, and it is hardware-aware, allowing for efficient computation on modern GPUs. The selection mechanism is key to Mamba's performance, as it enables the model to adapt to different input sequences and focus on relevant information. Mamba's performance is validated on various synthetic tasks, including selective copying and induction heads, as well as on real-world data such as DNA sequences and audio waveforms. The model's efficiency and effectiveness make it a promising candidate for general sequence modeling tasks.Mamba is a linear-time sequence modeling architecture that improves upon existing state space models (SSMs) by introducing a selection mechanism that allows the model to selectively propagate or forget information along the sequence length dimension. This mechanism enables the model to focus on relevant information and ignore irrelevant information, making it more effective at modeling discrete modalities such as language. Mamba is designed to be efficient and scalable, with linear scaling in sequence length and fast inference speeds. It achieves state-of-the-art performance across several modalities, including language, audio, and genomics. On language modeling, Mamba-3B outperforms Transformers of the same size and matches Transformers twice its size in both pretraining and downstream evaluation. Mamba is a simplified end-to-end neural network architecture without attention or even MLP blocks, and it is hardware-aware, allowing for efficient computation on modern GPUs. The selection mechanism is key to Mamba's performance, as it enables the model to adapt to different input sequences and focus on relevant information. Mamba's performance is validated on various synthetic tasks, including selective copying and induction heads, as well as on real-world data such as DNA sequences and audio waveforms. The model's efficiency and effectiveness make it a promising candidate for general sequence modeling tasks.