**Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models** **Authors:** Chengkai Liu **Abstract:** Sequential recommendation aims to estimate dynamic user preferences and sequential dependencies from historical user behaviors. While Transformer-based models have shown effectiveness, they suffer from quadratic computational complexity in attention operations, especially for long sequences. Inspired by state space models (SSMs), Mamba4Rec explores the potential of selective SSMs for efficient sequential recommendation. Building on the Mamba block, a selective SSM with an efficient hardware-aware parallel algorithm, Mamba4Rec incorporates techniques like position embedding, residual connections, layer normalization, and position-wise feed-forward networks to enhance performance while maintaining inference efficiency. Experiments on public datasets demonstrate Mamba4Rec's superior effectiveness and efficiency compared to RNN- and attention-based baselines. **Keywords:** Sequential Recommendation, State Space Models **Introduction:** Personalized online services rely on sequential recommendation systems to capture dynamic user preferences. Traditional methods like CNNs and RNNs suffer from issues such as catastrophic forgetting. Transformer-based models, while effective, have inference inefficiency due to quadratic attention complexity. SSMs, including Mamba, offer a solution by leveraging recurrent nature and structured state matrices to handle long-range dependencies efficiently. Mamba4Rec introduces a framework that combines Mamba blocks with position-wise feed-forward networks to capture both item-specific information and sequential context effectively. **Preliminaries:** The paper introduces the concept of SSMs, including the continuous and discrete forms, and the Mamba block, which operates on input sequences through linear projections, selective SSM, and a position-wise feed-forward network. The Mamba block's parameters are input-dependent, allowing it to selectively remember or forget information, enhancing its performance on long sequences. **Mamba4Rec:** Mamba4Rec is a sequential recommendation model that leverages Mamba blocks and position-wise feed-forward networks. It includes an embedding layer, Mamba layers, and a prediction layer. The model's architecture is flexible, allowing for single or stacked Mamba layers, each balancing effectiveness and efficiency. **Experiments:** Experiments on MovieLens-1M, Amazon-Beauty, and Amazon-Video-Games datasets show that Mamba4Rec outperforms various baselines in terms of effectiveness and efficiency. It achieves superior performance on sparse and dense datasets with varying sequence lengths, demonstrating its robustness and efficiency. **Conclusion:** Mamba4Rec addresses the limitations of Transformer-based models in sequential recommendation by leveraging selective SSMs. It shows strong performance and significant improvements in computational efficiency and memory cost, making it a promising approach for efficient sequential recommendation tasks. Future work aims to further optimize SSM-based models for recommendation systems.**Mamba4Rec: Towards Efficient Sequential Recommendation with Selective State Space Models** **Authors:** Chengkai Liu **Abstract:** Sequential recommendation aims to estimate dynamic user preferences and sequential dependencies from historical user behaviors. While Transformer-based models have shown effectiveness, they suffer from quadratic computational complexity in attention operations, especially for long sequences. Inspired by state space models (SSMs), Mamba4Rec explores the potential of selective SSMs for efficient sequential recommendation. Building on the Mamba block, a selective SSM with an efficient hardware-aware parallel algorithm, Mamba4Rec incorporates techniques like position embedding, residual connections, layer normalization, and position-wise feed-forward networks to enhance performance while maintaining inference efficiency. Experiments on public datasets demonstrate Mamba4Rec's superior effectiveness and efficiency compared to RNN- and attention-based baselines. **Keywords:** Sequential Recommendation, State Space Models **Introduction:** Personalized online services rely on sequential recommendation systems to capture dynamic user preferences. Traditional methods like CNNs and RNNs suffer from issues such as catastrophic forgetting. Transformer-based models, while effective, have inference inefficiency due to quadratic attention complexity. SSMs, including Mamba, offer a solution by leveraging recurrent nature and structured state matrices to handle long-range dependencies efficiently. Mamba4Rec introduces a framework that combines Mamba blocks with position-wise feed-forward networks to capture both item-specific information and sequential context effectively. **Preliminaries:** The paper introduces the concept of SSMs, including the continuous and discrete forms, and the Mamba block, which operates on input sequences through linear projections, selective SSM, and a position-wise feed-forward network. The Mamba block's parameters are input-dependent, allowing it to selectively remember or forget information, enhancing its performance on long sequences. **Mamba4Rec:** Mamba4Rec is a sequential recommendation model that leverages Mamba blocks and position-wise feed-forward networks. It includes an embedding layer, Mamba layers, and a prediction layer. The model's architecture is flexible, allowing for single or stacked Mamba layers, each balancing effectiveness and efficiency. **Experiments:** Experiments on MovieLens-1M, Amazon-Beauty, and Amazon-Video-Games datasets show that Mamba4Rec outperforms various baselines in terms of effectiveness and efficiency. It achieves superior performance on sparse and dense datasets with varying sequence lengths, demonstrating its robustness and efficiency. **Conclusion:** Mamba4Rec addresses the limitations of Transformer-based models in sequential recommendation by leveraging selective SSMs. It shows strong performance and significant improvements in computational efficiency and memory cost, making it a promising approach for efficient sequential recommendation tasks. Future work aims to further optimize SSM-based models for recommendation systems.