18 Mar 2024 | Yilong Ren, Yue Chen, Shuai Liu, Boyue Wang, Haiyang Yu, and Zhiyong Cui
The paper introduces TPLLM, a novel traffic prediction framework that leverages pre-trained Large Language Models (LLMs) to address the challenge of achieving accurate predictions with limited historical traffic data. The framework aims to exploit the few-shot learning capability and cross-modality knowledge transfer of LLMs, which are known for their effectiveness in handling diverse tasks with minimal data. TPLLM consists of two main components: a sequence embedding layer based on Convolutional Neural Networks (CNNs) and a graph embedding layer based on Graph Convolutional Networks (GCNs). These layers extract temporal and spatial features from traffic data, respectively, and integrate them into a format suitable for LLMs. The Low-Rank Adaptation (LoRA) fine-tuning approach is applied to TPLLM to reduce computational costs while maintaining performance. Experiments on real-world datasets demonstrate that TPLLM performs well in both full-sample and few-shot prediction scenarios, showcasing its effectiveness in traffic prediction tasks with limited historical data. The study also includes ablation experiments and sensitivity analysis to validate the contributions of each component and the optimal hyperparameters.The paper introduces TPLLM, a novel traffic prediction framework that leverages pre-trained Large Language Models (LLMs) to address the challenge of achieving accurate predictions with limited historical traffic data. The framework aims to exploit the few-shot learning capability and cross-modality knowledge transfer of LLMs, which are known for their effectiveness in handling diverse tasks with minimal data. TPLLM consists of two main components: a sequence embedding layer based on Convolutional Neural Networks (CNNs) and a graph embedding layer based on Graph Convolutional Networks (GCNs). These layers extract temporal and spatial features from traffic data, respectively, and integrate them into a format suitable for LLMs. The Low-Rank Adaptation (LoRA) fine-tuning approach is applied to TPLLM to reduce computational costs while maintaining performance. Experiments on real-world datasets demonstrate that TPLLM performs well in both full-sample and few-shot prediction scenarios, showcasing its effectiveness in traffic prediction tasks with limited historical data. The study also includes ablation experiments and sensitivity analysis to validate the contributions of each component and the optimal hyperparameters.