17 May 2024 | Lei Liu, Shuo Yu, Runze Wang, Zhenxun Ma, Yanming Shen
This paper proposes STG-LLM, an innovative approach to leverage Large Language Models (LLMs) for spatial-temporal forecasting. The key challenge is enabling LLMs to understand spatial-temporal data, which differs significantly from sequential text. To address this, the authors introduce two components: a spatial-temporal graph tokenizer (STG-Tokenizer) and a spatial-temporal graph adapter (STG-Adapter). The STG-Tokenizer converts spatial-temporal data into concise tokens that capture both spatial and temporal relationships. The STG-Adapter, consisting of linear encoding and decoding layers, bridges the gap between these tokens and LLM comprehension by fine-tuning a small set of parameters. This allows LLMs to understand the semantics of the tokens while preserving their original natural language understanding capabilities.
Extensive experiments on various spatial-temporal benchmark datasets show that STG-LLM achieves competitive performance, comparable to state-of-the-art methods. The approach effectively transforms spatial-temporal data into tokens that can be processed by LLMs, enabling accurate predictions. The STG-LLM is shown to be effective in tasks with limited data and can generalize well across different domains, including traffic, electricity, and finance. The method is also prompt-effective, as demonstrated by experiments where adding temporal prompts improved prediction accuracy. Ablation studies confirm the importance of each component, showing that the STG-Tokenizer and STG-Adapter are crucial for enabling LLMs to understand spatial-temporal data. The results indicate that STG-LLM requires only a small number of parameters to be fine-tuned, making it efficient and suitable for spatial-temporal forecasting tasks with data sparsity. Overall, STG-LLM demonstrates the potential of LLMs in spatial-temporal forecasting by effectively transforming and understanding spatial-temporal data.This paper proposes STG-LLM, an innovative approach to leverage Large Language Models (LLMs) for spatial-temporal forecasting. The key challenge is enabling LLMs to understand spatial-temporal data, which differs significantly from sequential text. To address this, the authors introduce two components: a spatial-temporal graph tokenizer (STG-Tokenizer) and a spatial-temporal graph adapter (STG-Adapter). The STG-Tokenizer converts spatial-temporal data into concise tokens that capture both spatial and temporal relationships. The STG-Adapter, consisting of linear encoding and decoding layers, bridges the gap between these tokens and LLM comprehension by fine-tuning a small set of parameters. This allows LLMs to understand the semantics of the tokens while preserving their original natural language understanding capabilities.
Extensive experiments on various spatial-temporal benchmark datasets show that STG-LLM achieves competitive performance, comparable to state-of-the-art methods. The approach effectively transforms spatial-temporal data into tokens that can be processed by LLMs, enabling accurate predictions. The STG-LLM is shown to be effective in tasks with limited data and can generalize well across different domains, including traffic, electricity, and finance. The method is also prompt-effective, as demonstrated by experiments where adding temporal prompts improved prediction accuracy. Ablation studies confirm the importance of each component, showing that the STG-Tokenizer and STG-Adapter are crucial for enabling LLMs to understand spatial-temporal data. The results indicate that STG-LLM requires only a small number of parameters to be fine-tuned, making it efficient and suitable for spatial-temporal forecasting tasks with data sparsity. Overall, STG-LLM demonstrates the potential of LLMs in spatial-temporal forecasting by effectively transforming and understanding spatial-temporal data.