UrbanGPT: Spatio-Temporal Large Language Models

UrbanGPT: Spatio-Temporal Large Language Models

2024 | Zhonghang Li, Lianghao Xia, Jiabin Tang, Yong Xu, Lei Shi, Long Xia, Dawei Yin and Chao Huang
UrbanGPT is a spatio-temporal large language model (LLM) designed to generalize across diverse urban tasks, particularly in scenarios with limited labeled data. The model integrates a spatio-temporal dependency encoder with the instruction-tuning paradigm, enabling it to understand complex spatio-temporal dependencies. This approach allows UrbanGPT to generate more accurate predictions even when data is scarce. The model was evaluated on various public datasets, including NYC-taxi, NYC-bike, NYC-crime, and CHI-taxi, demonstrating superior performance compared to state-of-the-art baselines. UrbanGPT outperforms existing models in zero-shot scenarios, where labeled data is limited, and shows strong generalization capabilities across different spatio-temporal prediction tasks. The model's architecture includes a spatio-temporal encoder that captures temporal dynamics and a lightweight alignment module that integrates textual and spatio-temporal information. The regression layer in the instruction-tuning process enables the model to generate precise numerical predictions. UrbanGPT's performance was validated through extensive experiments, showing its effectiveness in predicting traffic flow, crime rates, and other urban phenomena. The model's ability to handle sparse data and its robustness in cross-city prediction tasks highlight its potential for real-world applications in urban computing. The study also addresses the challenges of data scarcity in urban sensing scenarios and proposes a novel approach to enhance the generalization capabilities of spatio-temporal models. The results demonstrate that UrbanGPT can effectively capture spatio-temporal patterns and make accurate predictions in previously unseen environments.UrbanGPT is a spatio-temporal large language model (LLM) designed to generalize across diverse urban tasks, particularly in scenarios with limited labeled data. The model integrates a spatio-temporal dependency encoder with the instruction-tuning paradigm, enabling it to understand complex spatio-temporal dependencies. This approach allows UrbanGPT to generate more accurate predictions even when data is scarce. The model was evaluated on various public datasets, including NYC-taxi, NYC-bike, NYC-crime, and CHI-taxi, demonstrating superior performance compared to state-of-the-art baselines. UrbanGPT outperforms existing models in zero-shot scenarios, where labeled data is limited, and shows strong generalization capabilities across different spatio-temporal prediction tasks. The model's architecture includes a spatio-temporal encoder that captures temporal dynamics and a lightweight alignment module that integrates textual and spatio-temporal information. The regression layer in the instruction-tuning process enables the model to generate precise numerical predictions. UrbanGPT's performance was validated through extensive experiments, showing its effectiveness in predicting traffic flow, crime rates, and other urban phenomena. The model's ability to handle sparse data and its robustness in cross-city prediction tasks highlight its potential for real-world applications in urban computing. The study also addresses the challenges of data scarcity in urban sensing scenarios and proposes a novel approach to enhance the generalization capabilities of spatio-temporal models. The results demonstrate that UrbanGPT can effectively capture spatio-temporal patterns and make accurate predictions in previously unseen environments.
Reach us at info@study.space
[slides] UrbanGPT%3A Spatio-Temporal Large Language Models | StudySpace