The paper introduces a novel perspective on Mamba models, showing that they can be viewed as attention-driven models. Mamba models are selective state space models (SSMs) that efficiently model multiple domains, including NLP, computer vision, and more. They are trained in parallel on the entire sequence and deployed in an autoregressive manner. The authors demonstrate that Mamba models can be interpreted as attention mechanisms, enabling empirical and theoretical comparisons with self-attention layers in transformers. This perspective allows for the use of explainability methods to understand the inner workings of Mamba models. The paper shows that Mamba models generate significantly more attention matrices than transformers, which is a key insight for understanding their behavior. The authors develop explainability and interpretability tools based on these hidden attention matrices, enabling the analysis of Mamba models in a way similar to transformers. They also show that Mamba-based attention maps have comparable explainability metrics to those of transformers. The paper presents a reformulation of Mamba computation using a data-control linear operator, revealing hidden attention matrices within the Mamba layer. This enables the use of established interpretability techniques to develop tools for interpreting Mamba models. The authors also provide a theoretical analysis of the evolution of attention capabilities in state-space models, offering a deeper understanding of the factors contributing to Mamba's effectiveness. The paper includes experiments on the explainability of Mamba models, comparing them with transformers. The results show that Mamba-based heatmaps are often more complete than their transformer-based counterparts. The paper also discusses the potential of Mamba's attention mechanism in achieving high levels of in-context learning, which is a key capability of transformers. The paper concludes that Mamba models can be reformulated as an implicit form of causal self-attention mechanism, linking them directly with transformer layers. The attention perspective plays a role in understanding the inner representation of the Mamba model. The authors introduce the first explainability techniques for Mamba models, for both task-specific and task-agnostic regimes. This contribution equips the research community with novel tools for examining the performance, fairness, robustness, and weaknesses of Mamba models, paving the way for future improvements and enabling weakly supervised downstream tasks.The paper introduces a novel perspective on Mamba models, showing that they can be viewed as attention-driven models. Mamba models are selective state space models (SSMs) that efficiently model multiple domains, including NLP, computer vision, and more. They are trained in parallel on the entire sequence and deployed in an autoregressive manner. The authors demonstrate that Mamba models can be interpreted as attention mechanisms, enabling empirical and theoretical comparisons with self-attention layers in transformers. This perspective allows for the use of explainability methods to understand the inner workings of Mamba models. The paper shows that Mamba models generate significantly more attention matrices than transformers, which is a key insight for understanding their behavior. The authors develop explainability and interpretability tools based on these hidden attention matrices, enabling the analysis of Mamba models in a way similar to transformers. They also show that Mamba-based attention maps have comparable explainability metrics to those of transformers. The paper presents a reformulation of Mamba computation using a data-control linear operator, revealing hidden attention matrices within the Mamba layer. This enables the use of established interpretability techniques to develop tools for interpreting Mamba models. The authors also provide a theoretical analysis of the evolution of attention capabilities in state-space models, offering a deeper understanding of the factors contributing to Mamba's effectiveness. The paper includes experiments on the explainability of Mamba models, comparing them with transformers. The results show that Mamba-based heatmaps are often more complete than their transformer-based counterparts. The paper also discusses the potential of Mamba's attention mechanism in achieving high levels of in-context learning, which is a key capability of transformers. The paper concludes that Mamba models can be reformulated as an implicit form of causal self-attention mechanism, linking them directly with transformer layers. The attention perspective plays a role in understanding the inner representation of the Mamba model. The authors introduce the first explainability techniques for Mamba models, for both task-specific and task-agnostic regimes. This contribution equips the research community with novel tools for examining the performance, fairness, robustness, and weaknesses of Mamba models, paving the way for future improvements and enabling weakly supervised downstream tasks.