SiMBA is a novel architecture that improves upon state-of-the-art state space models (SSMs) for vision and multivariate time series tasks. It introduces Einstein FFT (EinFFT) for channel modeling and uses Mamba for sequence modeling. The architecture addresses the stability issues of Mamba when scaled to large networks, particularly on the ImageNet dataset. Extensive performance studies across image and time-series benchmarks show that SiMBA outperforms existing SSMs and bridges the performance gap with state-of-the-art transformers. SiMBA achieves state-of-the-art results on ImageNet, transfer learning benchmarks like Stanford Car and Flower, and seven time series datasets. The architecture also demonstrates strong performance in object detection and instance segmentation tasks. SiMBA combines Mamba for sequence modeling with EinFFT for channel modeling, offering a more efficient and stable solution compared to traditional methods. The paper also presents ablation studies showing the effectiveness of different components of SiMBA, including residual connections and dropouts. Overall, SiMBA provides a versatile and effective solution for a wide range of sequence modeling tasks.SiMBA is a novel architecture that improves upon state-of-the-art state space models (SSMs) for vision and multivariate time series tasks. It introduces Einstein FFT (EinFFT) for channel modeling and uses Mamba for sequence modeling. The architecture addresses the stability issues of Mamba when scaled to large networks, particularly on the ImageNet dataset. Extensive performance studies across image and time-series benchmarks show that SiMBA outperforms existing SSMs and bridges the performance gap with state-of-the-art transformers. SiMBA achieves state-of-the-art results on ImageNet, transfer learning benchmarks like Stanford Car and Flower, and seven time series datasets. The architecture also demonstrates strong performance in object detection and instance segmentation tasks. SiMBA combines Mamba for sequence modeling with EinFFT for channel modeling, offering a more efficient and stable solution compared to traditional methods. The paper also presents ablation studies showing the effectiveness of different components of SiMBA, including residual connections and dropouts. Overall, SiMBA provides a versatile and effective solution for a wide range of sequence modeling tasks.