SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters

SparseTSF: Modeling Long-term Time Series Forecasting with 1k Parameters

2024 | Shengsheng Lin, Weiwei Lin, Wentai Wu, Haojun Chen, Junjie Yang
This paper introduces SparseTSF, a novel and extremely lightweight model for Long-term Time Series Forecasting (LTSF). The core of SparseTSF is the Cross-Period Sparse Forecasting technique, which simplifies the forecasting task by decoupling the periodicity and trend in time series data. This technique involves downsampling the original sequences to focus on cross-period trend prediction, effectively extracting periodic features while minimizing the model's complexity and parameter count. Based on this technique, the SparseTSF model uses fewer than 1k parameters to achieve competitive or superior performance compared to state-of-the-art models. The model also demonstrates remarkable generalization capabilities, making it well-suited for scenarios with limited computational resources, small samples, or low-quality data. The paper provides a detailed theoretical analysis of the SparseTSF model, including its parameter efficiency and effectiveness. Experimental results on four mainstream LTSF datasets show that SparseTSF ranks within the top two in all scenarios, achieving or closely approaching state-of-the-art levels with a significantly smaller parameter scale. The model's extreme lightweight nature is further demonstrated through static and runtime metrics, where it outperforms other models in terms of parameters, MACs, maximum memory usage, and training duration. Ablation studies and analysis reveal the effectiveness of the Sparse technique in enhancing the performance of base models, including linear, transformer, and GRU models. The model's ability to extract periodic features is also visualized and analyzed, showing that SparseTSF learns more distinct evenly spaced weight distribution stripes compared to the linear model. Additionally, the impact of the hyperparameter \( w \) on the model's performance is explored, with optimal results observed when \( w \) aligns with the data's intrinsic main period. The generalization capabilities of SparseTSF are further investigated, demonstrating its robustness in transferring to different datasets with the same principal periodicity. The model's performance is compared with existing methods, highlighting its unique contributions and differences. Finally, the paper discusses the limitations and future work, including the challenges of handling ultra-long periods and multiple periods, and aims to address these by designing additional modules to enhance the model's performance and parameter efficiency.This paper introduces SparseTSF, a novel and extremely lightweight model for Long-term Time Series Forecasting (LTSF). The core of SparseTSF is the Cross-Period Sparse Forecasting technique, which simplifies the forecasting task by decoupling the periodicity and trend in time series data. This technique involves downsampling the original sequences to focus on cross-period trend prediction, effectively extracting periodic features while minimizing the model's complexity and parameter count. Based on this technique, the SparseTSF model uses fewer than 1k parameters to achieve competitive or superior performance compared to state-of-the-art models. The model also demonstrates remarkable generalization capabilities, making it well-suited for scenarios with limited computational resources, small samples, or low-quality data. The paper provides a detailed theoretical analysis of the SparseTSF model, including its parameter efficiency and effectiveness. Experimental results on four mainstream LTSF datasets show that SparseTSF ranks within the top two in all scenarios, achieving or closely approaching state-of-the-art levels with a significantly smaller parameter scale. The model's extreme lightweight nature is further demonstrated through static and runtime metrics, where it outperforms other models in terms of parameters, MACs, maximum memory usage, and training duration. Ablation studies and analysis reveal the effectiveness of the Sparse technique in enhancing the performance of base models, including linear, transformer, and GRU models. The model's ability to extract periodic features is also visualized and analyzed, showing that SparseTSF learns more distinct evenly spaced weight distribution stripes compared to the linear model. Additionally, the impact of the hyperparameter \( w \) on the model's performance is explored, with optimal results observed when \( w \) aligns with the data's intrinsic main period. The generalization capabilities of SparseTSF are further investigated, demonstrating its robustness in transferring to different datasets with the same principal periodicity. The model's performance is compared with existing methods, highlighting its unique contributions and differences. Finally, the paper discusses the limitations and future work, including the challenges of handling ultra-long periods and multiple periods, and aims to address these by designing additional modules to enhance the model's performance and parameter efficiency.
Reach us at info@study.space