**UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction** **Authors:** Lan Feng, Mohammadhossein Bahari, Kaouther Messaoud Ben Amor, Éloi Zablocki, Matthieu Cord, and Alexandre Alahi **Abstract:** Vehicle trajectory prediction has become increasingly reliant on data-driven solutions, but their ability to scale across different data domains and the impact of larger dataset sizes on generalization remain underexplored. To address these challenges, the authors introduce UniTraj, a comprehensive framework that unifies various datasets, models, and evaluation criteria. UniTraj enables extensive experiments, revealing that model performance significantly drops when transferred to other datasets. However, increasing data size and diversity can substantially improve performance, leading to a new state-of-the-art result for the nuScenes dataset. The framework provides insights into dataset characteristics and is available as an open-source tool. **Keywords:** Vehicle trajectory prediction · Multi-dataset framework · Domain generalization **Introduction:** Predicting vehicle trajectories is crucial for autonomous driving systems. While deep learning models can achieve high accuracy, they are heavily reliant on specific training data domains. This paper investigates two key research questions: (RQ1) the performance drop of trajectory prediction models when transferred to new domains, and (RQ2) the impact of increasing dataset sizes on model performance. The authors address these questions by leveraging multiple trajectory prediction datasets, which provide diverse domains and enable the exploration of asymptotic limits of data scaling. **UniTraj Framework:** UniTraj consists of three main components: unified data, unified models, and unified evaluation. The unified data component addresses discrepancies in data formats and features, while the unified models component integrates state-of-the-art trajectory prediction models. The unified evaluation component provides a comprehensive set of metrics for consistent evaluation across different datasets. **Experiments:** The experiments highlight the importance of geographical diversity in data collection and the benefits of larger, more diverse datasets. The results show that models trained on combined datasets outperform those trained on single datasets, particularly on the nuScenes dataset. The framework also enables cross-dataset and cross-city generalization studies, revealing significant performance gaps between datasets and cities. **Conclusion:** The study underscores the need for larger, more diverse datasets in vehicle trajectory prediction. The UniTraj framework, with its comprehensive resource, will significantly advance research in this field.**UniTraj: A Unified Framework for Scalable Vehicle Trajectory Prediction** **Authors:** Lan Feng, Mohammadhossein Bahari, Kaouther Messaoud Ben Amor, Éloi Zablocki, Matthieu Cord, and Alexandre Alahi **Abstract:** Vehicle trajectory prediction has become increasingly reliant on data-driven solutions, but their ability to scale across different data domains and the impact of larger dataset sizes on generalization remain underexplored. To address these challenges, the authors introduce UniTraj, a comprehensive framework that unifies various datasets, models, and evaluation criteria. UniTraj enables extensive experiments, revealing that model performance significantly drops when transferred to other datasets. However, increasing data size and diversity can substantially improve performance, leading to a new state-of-the-art result for the nuScenes dataset. The framework provides insights into dataset characteristics and is available as an open-source tool. **Keywords:** Vehicle trajectory prediction · Multi-dataset framework · Domain generalization **Introduction:** Predicting vehicle trajectories is crucial for autonomous driving systems. While deep learning models can achieve high accuracy, they are heavily reliant on specific training data domains. This paper investigates two key research questions: (RQ1) the performance drop of trajectory prediction models when transferred to new domains, and (RQ2) the impact of increasing dataset sizes on model performance. The authors address these questions by leveraging multiple trajectory prediction datasets, which provide diverse domains and enable the exploration of asymptotic limits of data scaling. **UniTraj Framework:** UniTraj consists of three main components: unified data, unified models, and unified evaluation. The unified data component addresses discrepancies in data formats and features, while the unified models component integrates state-of-the-art trajectory prediction models. The unified evaluation component provides a comprehensive set of metrics for consistent evaluation across different datasets. **Experiments:** The experiments highlight the importance of geographical diversity in data collection and the benefits of larger, more diverse datasets. The results show that models trained on combined datasets outperform those trained on single datasets, particularly on the nuScenes dataset. The framework also enables cross-dataset and cross-city generalization studies, revealing significant performance gaps between datasets and cities. **Conclusion:** The study underscores the need for larger, more diverse datasets in vehicle trajectory prediction. The UniTraj framework, with its comprehensive resource, will significantly advance research in this field.