ContiFormer is a Continuous-Time Transformer designed for modeling irregular time series data. It extends the relation modeling of vanilla Transformers to the continuous-time domain by integrating the modeling capabilities of continuous dynamics from Neural ODEs with the attention mechanism of Transformers. This approach allows ContiFormer to capture intricate correlations within irregular time series, which traditional methods struggle with due to their discrete nature. The model mathematically characterizes its expressive power and shows that various Transformer variants can be viewed as special cases of ContiFormer. Extensive experiments on both synthetic and real-world datasets demonstrate that ContiFormer outperforms existing methods in modeling and predicting irregular time series. The model's continuous-time attention mechanism enables it to handle dynamic changes in the system over time, making it suitable for tasks such as interpolation, classification, and forecasting. ContiFormer also benefits from the parallelism of the Transformer architecture, allowing efficient computation and better performance in capturing complex continuous-time dynamics. The model's ability to handle irregular time series makes it a versatile tool for applications in healthcare, finance, and other domains where data is not uniformly sampled.ContiFormer is a Continuous-Time Transformer designed for modeling irregular time series data. It extends the relation modeling of vanilla Transformers to the continuous-time domain by integrating the modeling capabilities of continuous dynamics from Neural ODEs with the attention mechanism of Transformers. This approach allows ContiFormer to capture intricate correlations within irregular time series, which traditional methods struggle with due to their discrete nature. The model mathematically characterizes its expressive power and shows that various Transformer variants can be viewed as special cases of ContiFormer. Extensive experiments on both synthetic and real-world datasets demonstrate that ContiFormer outperforms existing methods in modeling and predicting irregular time series. The model's continuous-time attention mechanism enables it to handle dynamic changes in the system over time, making it suitable for tasks such as interpolation, classification, and forecasting. ContiFormer also benefits from the parallelism of the Transformer architecture, allowing efficient computation and better performance in capturing complex continuous-time dynamics. The model's ability to handle irregular time series makes it a versatile tool for applications in healthcare, finance, and other domains where data is not uniformly sampled.