**UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction** **Authors:** Yuan Yuan, Jingtao Ding, Jie Feng, Depeng Jin, Yong Li **Affiliation:** Department of Electronic Engineering, Tsinghua University, Beijing, China **Abstract:** Urban spatio-temporal prediction is crucial for informed decision-making in areas such as traffic management, resource optimization, and emergency response. Despite significant advancements in pre-trained natural language models, a universal solution for spatio-temporal prediction remains challenging. Existing approaches are typically tailored for specific scenarios, requiring task-specific designs and extensive domain-specific training data. This study introduces UniST, a universal model designed for general urban spatio-temporal prediction across a wide range of scenarios. UniST leverages diverse spatio-temporal data from different scenarios, effectively captures spatio-temporal dynamics through pre-training, and enhances generalization capabilities using knowledge-guided prompts. Extensive experiments on over 20 spatio-temporal datasets demonstrate UniST's efficacy, particularly in few-shot and zero-shot prediction tasks. The datasets and code implementation are available on GitHub. **Key Concepts:** - Computing methodologies → Machine learning approaches **Keywords:** Spatio-temporal prediction, prompt learning, universal model **Contributions:** - UniST is the first attempt to address universal spatio-temporal prediction by investigating the potential of a one-for-all model in diverse scenarios. - UniST harnesses data diversity and achieves universal spatio-temporal prediction through advanced pre-training and prompt learning. - UniST demonstrates superior performance and generalization capabilities in various prediction tasks, especially in few-shot and zero-shot settings. **Methodology:** - **Preliminary:** UniST uses a grid system for spatial partitioning and transforms spatio-temporal data into a unified sequential format using spatio-temporal patching. - **Pre-training and Prompt Learning:** UniST employs large-scale spatio-temporal pre-training and spatio-temporal knowledge-guided prompt learning to handle diverse data distributions. - **Base Model:** UniST uses a Transformer-based encoder-decoder architecture with spatio-temporal patching and positional encoding. - **Spatio-Temporal Self-Supervised Pre-train:** UniST introduces four distinct masking strategies to capture complex spatio-temporal relationships. - **Spatio-Temporal Knowledge-Guided Prompt:** UniST leverages spatio-temporal domain knowledge to generate prompts that align with shared patterns across different datasets. **Performance Evaluations:** - **Short-Term Prediction:** UniST outperforms all baselines across various datasets. - **Long-Term Prediction:** UniST consistently outperforms baselines even with a more extended prediction horizon. - **Few-Shot Prediction:** UniST shows superior performance in few-shot learning, achieving a larger relative improvement compared to baselines. - **Zero-Shot Prediction:** UniST achieves remarkable**UniST: A Prompt-Empowered Universal Model for Urban Spatio-Temporal Prediction** **Authors:** Yuan Yuan, Jingtao Ding, Jie Feng, Depeng Jin, Yong Li **Affiliation:** Department of Electronic Engineering, Tsinghua University, Beijing, China **Abstract:** Urban spatio-temporal prediction is crucial for informed decision-making in areas such as traffic management, resource optimization, and emergency response. Despite significant advancements in pre-trained natural language models, a universal solution for spatio-temporal prediction remains challenging. Existing approaches are typically tailored for specific scenarios, requiring task-specific designs and extensive domain-specific training data. This study introduces UniST, a universal model designed for general urban spatio-temporal prediction across a wide range of scenarios. UniST leverages diverse spatio-temporal data from different scenarios, effectively captures spatio-temporal dynamics through pre-training, and enhances generalization capabilities using knowledge-guided prompts. Extensive experiments on over 20 spatio-temporal datasets demonstrate UniST's efficacy, particularly in few-shot and zero-shot prediction tasks. The datasets and code implementation are available on GitHub. **Key Concepts:** - Computing methodologies → Machine learning approaches **Keywords:** Spatio-temporal prediction, prompt learning, universal model **Contributions:** - UniST is the first attempt to address universal spatio-temporal prediction by investigating the potential of a one-for-all model in diverse scenarios. - UniST harnesses data diversity and achieves universal spatio-temporal prediction through advanced pre-training and prompt learning. - UniST demonstrates superior performance and generalization capabilities in various prediction tasks, especially in few-shot and zero-shot settings. **Methodology:** - **Preliminary:** UniST uses a grid system for spatial partitioning and transforms spatio-temporal data into a unified sequential format using spatio-temporal patching. - **Pre-training and Prompt Learning:** UniST employs large-scale spatio-temporal pre-training and spatio-temporal knowledge-guided prompt learning to handle diverse data distributions. - **Base Model:** UniST uses a Transformer-based encoder-decoder architecture with spatio-temporal patching and positional encoding. - **Spatio-Temporal Self-Supervised Pre-train:** UniST introduces four distinct masking strategies to capture complex spatio-temporal relationships. - **Spatio-Temporal Knowledge-Guided Prompt:** UniST leverages spatio-temporal domain knowledge to generate prompts that align with shared patterns across different datasets. **Performance Evaluations:** - **Short-Term Prediction:** UniST outperforms all baselines across various datasets. - **Long-Term Prediction:** UniST consistently outperforms baselines even with a more extended prediction horizon. - **Few-Shot Prediction:** UniST shows superior performance in few-shot learning, achieving a larger relative improvement compared to baselines. - **Zero-Shot Prediction:** UniST achieves remarkable