HDMixer: Hierarchical Dependency with Extendable Patch for Multivariate Time Series Forecasting

HDMixer: Hierarchical Dependency with Extendable Patch for Multivariate Time Series Forecasting

2024 | Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shouhong Ding, Zhengyang Zhou, Yang Wang
**HDMixer: Hierarchical Dependency with Extendable Patch for Multivariate Time Series Forecasting** **Authors:** Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shoubong Ding, Zhengyang Zhou, Yang Wang **Institution:** University of Science and Technology of China, Suzhou Institute for Advanced Research, Johns Hopkins University, Youtu Laboratory, Tencent, State Key Laboratory of Resources and Environmental Information System **Abstract:** Multivariate time series (MTS) prediction has gained widespread adoption in various applications. Recent methods have employed patching to enhance local semantics and improve model performance, but length-fixed patches often lose temporal boundary information and focus primarily on long-term dependencies. To address these challenges, we propose HDMixer, a pure MLP-based model that introduces a Length-Extendable Patcher (LEP) to enrich patch boundary information and a Hierarchical Dependency Explorer (HDE) to model hierarchical interactions. LEP adaptively extends patch lengths based on temporal characteristics, while HDE captures short-term and long-term dependencies within and across patches. Extensive experiments on nine real-world datasets demonstrate HDMixer's superior performance, achieving top-1 results in 59 settings and top-2 in 13 settings. **Introduction:** Multivariate time series prediction is crucial in various applications, such as weather forecasting and economic assessment. Deep learning has significantly improved prediction performance, but existing methods often use length-fixed patches, leading to issues like loss of boundary information and semantic incoherence. HDMixer addresses these challenges by introducing LEP and HDE, which enhance local semantics and model hierarchical interactions efficiently. **Methodology:** HDMixer is designed to learn length-extendable patches and capture interactions across different dimensions. LEP adaptively expands patch lengths using bi-linear interpolation, ensuring more complete semantic patch partitioning. HDE, composed of multiple stacked Mixers, captures short-term, long-term, and cross-variable dependencies. The model's efficiency and effectiveness are demonstrated through extensive experiments on real-world datasets. **Results:** HDMixer outperforms state-of-the-art methods, achieving significant reductions in MSE and MAE across multiple datasets and prediction lengths. Ablation studies and hyperparameter sensitivity analyses further validate the effectiveness of HDMixer's components and settings. **Conclusion:** HDMixer effectively addresses the challenges of MTS forecasting by enhancing local semantics and modeling hierarchical interactions. Its superior performance and efficiency make it a promising approach for various real-world applications.**HDMixer: Hierarchical Dependency with Extendable Patch for Multivariate Time Series Forecasting** **Authors:** Qihe Huang, Lei Shen, Ruixin Zhang, Jiahuan Cheng, Shoubong Ding, Zhengyang Zhou, Yang Wang **Institution:** University of Science and Technology of China, Suzhou Institute for Advanced Research, Johns Hopkins University, Youtu Laboratory, Tencent, State Key Laboratory of Resources and Environmental Information System **Abstract:** Multivariate time series (MTS) prediction has gained widespread adoption in various applications. Recent methods have employed patching to enhance local semantics and improve model performance, but length-fixed patches often lose temporal boundary information and focus primarily on long-term dependencies. To address these challenges, we propose HDMixer, a pure MLP-based model that introduces a Length-Extendable Patcher (LEP) to enrich patch boundary information and a Hierarchical Dependency Explorer (HDE) to model hierarchical interactions. LEP adaptively extends patch lengths based on temporal characteristics, while HDE captures short-term and long-term dependencies within and across patches. Extensive experiments on nine real-world datasets demonstrate HDMixer's superior performance, achieving top-1 results in 59 settings and top-2 in 13 settings. **Introduction:** Multivariate time series prediction is crucial in various applications, such as weather forecasting and economic assessment. Deep learning has significantly improved prediction performance, but existing methods often use length-fixed patches, leading to issues like loss of boundary information and semantic incoherence. HDMixer addresses these challenges by introducing LEP and HDE, which enhance local semantics and model hierarchical interactions efficiently. **Methodology:** HDMixer is designed to learn length-extendable patches and capture interactions across different dimensions. LEP adaptively expands patch lengths using bi-linear interpolation, ensuring more complete semantic patch partitioning. HDE, composed of multiple stacked Mixers, captures short-term, long-term, and cross-variable dependencies. The model's efficiency and effectiveness are demonstrated through extensive experiments on real-world datasets. **Results:** HDMixer outperforms state-of-the-art methods, achieving significant reductions in MSE and MAE across multiple datasets and prediction lengths. Ablation studies and hyperparameter sensitivity analyses further validate the effectiveness of HDMixer's components and settings. **Conclusion:** HDMixer effectively addresses the challenges of MTS forecasting by enhancing local semantics and modeling hierarchical interactions. Its superior performance and efficiency make it a promising approach for various real-world applications.
Reach us at info@study.space