The paper introduces ClinicalMamba, a specialized version of the Mamba language model designed to handle longitudinal clinical notes. ClinicalMamba is pre-trained on a large corpus of clinical notes, addressing the unique linguistic characteristics and information processing needs of the medical domain. The model, with 130 million and 2.8 billion parameters, demonstrates superior performance in modeling clinical language across extended text lengths compared to other models. Through few-shot learning, ClinicalMamba outperforms existing clinical language models and large language models like GPT-4 in longitudinal clinical tasks, achieving notable benchmarks in speed and performance. The study also highlights the model's ability to handle long context lengths, which is crucial for tasks such as predicting disease progression and extracting medical relations. The paper includes a detailed evaluation of ClinicalMamba on cohort selection and ICD coding tasks, showing its effectiveness in handling complex clinical narratives and improving the efficiency of clinical data processing.The paper introduces ClinicalMamba, a specialized version of the Mamba language model designed to handle longitudinal clinical notes. ClinicalMamba is pre-trained on a large corpus of clinical notes, addressing the unique linguistic characteristics and information processing needs of the medical domain. The model, with 130 million and 2.8 billion parameters, demonstrates superior performance in modeling clinical language across extended text lengths compared to other models. Through few-shot learning, ClinicalMamba outperforms existing clinical language models and large language models like GPT-4 in longitudinal clinical tasks, achieving notable benchmarks in speed and performance. The study also highlights the model's ability to handle long context lengths, which is crucial for tasks such as predicting disease progression and extracting medical relations. The paper includes a detailed evaluation of ClinicalMamba on cohort selection and ICD coding tasks, showing its effectiveness in handling complex clinical narratives and improving the efficiency of clinical data processing.