2024 | Jiaxiang Dong * 1 Haixu Wu * 1 Yuxuan Wang 1 Yunzhong Qiu 1 Li Zhang 1 Jianmin Wang 1 Mingsheng Long 1
TimeSiam is a novel self-supervised pre-training framework designed for time series modeling. It leverages Siamese networks to capture temporal correlations between randomly sampled past and current subseries, enhancing the model's ability to learn diverse time-dependent representations. The framework uses simple data augmentation techniques, such as masking, to improve the diversity and distinctiveness of Siamese subseries pairs, enforcing the encoder to learn temporally related information. Learnable lineage embeddings are introduced to distinguish temporal distances between subseries, further enriching the model's capacity for learning diverse temporal correlations. TimeSiam consistently outperforms existing advanced pre-training baselines across 13 standard benchmarks in both intra- and cross-domain scenarios, demonstrating superior forecasting and classification capabilities. The code for TimeSiam is available at <https://github.com/thuml/TimeSiam>.TimeSiam is a novel self-supervised pre-training framework designed for time series modeling. It leverages Siamese networks to capture temporal correlations between randomly sampled past and current subseries, enhancing the model's ability to learn diverse time-dependent representations. The framework uses simple data augmentation techniques, such as masking, to improve the diversity and distinctiveness of Siamese subseries pairs, enforcing the encoder to learn temporally related information. Learnable lineage embeddings are introduced to distinguish temporal distances between subseries, further enriching the model's capacity for learning diverse temporal correlations. TimeSiam consistently outperforms existing advanced pre-training baselines across 13 standard benchmarks in both intra- and cross-domain scenarios, demonstrating superior forecasting and classification capabilities. The code for TimeSiam is available at <https://github.com/thuml/TimeSiam>.