ClinicalBERT is a model designed to process and predict 30-day hospital readmissions using clinical notes. The authors, Kexin Huang, Jaan Altosaar, and Rajesh Ranganath, aim to develop a continuous representation of clinical notes, which are often underutilized due to their high dimensionality and sparsity. They apply BERT, a transformer-based model, to clinical text, pre-training it on clinical notes and fine-tuning it for readmission prediction. ClinicalBERT outperforms various baselines on both discharge summaries and early notes in the intensive care unit, demonstrating high-quality relationships between medical concepts. The model's attention weights can be used to interpret predictions, and the parameters are open-sourced for further research. ClinicalBERT is a flexible framework that can be adapted to other clinical predictive tasks, improving upon previous methods for processing clinical text.ClinicalBERT is a model designed to process and predict 30-day hospital readmissions using clinical notes. The authors, Kexin Huang, Jaan Altosaar, and Rajesh Ranganath, aim to develop a continuous representation of clinical notes, which are often underutilized due to their high dimensionality and sparsity. They apply BERT, a transformer-based model, to clinical text, pre-training it on clinical notes and fine-tuning it for readmission prediction. ClinicalBERT outperforms various baselines on both discharge summaries and early notes in the intensive care unit, demonstrating high-quality relationships between medical concepts. The model's attention weights can be used to interpret predictions, and the parameters are open-sourced for further research. ClinicalBERT is a flexible framework that can be adapted to other clinical predictive tasks, improving upon previous methods for processing clinical text.