2024-03-18 | Yilong Ren, Yue Chen, Shuai Liu, Boyue Wang, Haiyang Yu, and Zhiyong Cui
TPLLM is a traffic prediction framework based on pretrained large language models (LLMs). Traffic prediction is crucial for effective traffic management, but existing deep learning models require large datasets, which are often unavailable. TPLLM leverages LLMs' strengths in cross-modal knowledge transfer and few-shot learning. It uses Convolutional Neural Networks (CNNs) and Graph Convolutional Networks (GCNs) to extract temporal and spatial features from traffic data, which are then integrated for LLM processing. A Low-Rank Adaptation (LoRA) fine-tuning approach is applied to reduce computational costs and improve efficiency. Experiments on two real-world datasets show that TPLLM performs well in both full-sample and few-shot prediction scenarios, making it effective for traffic prediction in areas with limited historical data. The framework includes an embedding module that combines sequence and graph features, and uses LoRA to fine-tune the LLM. The results demonstrate that TPLLM can achieve accurate predictions with limited data, leveraging the prior knowledge of pretrained LLMs. The framework is evaluated using metrics like MAE, RMSE, and MAPE, and ablation studies confirm the effectiveness of each component. TPLLM outperforms existing methods in both full-sample and few-shot prediction, showing its potential for real-world traffic management.TPLLM is a traffic prediction framework based on pretrained large language models (LLMs). Traffic prediction is crucial for effective traffic management, but existing deep learning models require large datasets, which are often unavailable. TPLLM leverages LLMs' strengths in cross-modal knowledge transfer and few-shot learning. It uses Convolutional Neural Networks (CNNs) and Graph Convolutional Networks (GCNs) to extract temporal and spatial features from traffic data, which are then integrated for LLM processing. A Low-Rank Adaptation (LoRA) fine-tuning approach is applied to reduce computational costs and improve efficiency. Experiments on two real-world datasets show that TPLLM performs well in both full-sample and few-shot prediction scenarios, making it effective for traffic prediction in areas with limited historical data. The framework includes an embedding module that combines sequence and graph features, and uses LoRA to fine-tune the LLM. The results demonstrate that TPLLM can achieve accurate predictions with limited data, leveraging the prior knowledge of pretrained LLMs. The framework is evaluated using metrics like MAE, RMSE, and MAPE, and ablation studies confirm the effectiveness of each component. TPLLM outperforms existing methods in both full-sample and few-shot prediction, showing its potential for real-world traffic management.