TransGPT: Multi-modal Generative Pre-trained Transformer for Transportation

TransGPT: Multi-modal Generative Pre-trained Transformer for Transportation

2024 | Peng Wang, Xiang Wei, Fangxu Hu, Wenjuan Han
TransGPT is a novel multi-modal large language model designed for the transportation domain, consisting of two variants: TransGPT-SM for single-modal data and TransGPT-MM for multi-modal data. TransGPT-SM is fine-tuned on a single-modal transportation dataset (STD) containing 12.5 million tokens of textual data from various transportation sources. TransGPT-MM is fine-tuned on a multi-modal transportation dataset (MTD) manually collected from three areas: driving tests, traffic signs, and landmarks. The MTD includes aligned images and texts, with texts divided into questions and answers. TransGPT is evaluated on benchmark datasets for various transportation tasks, outperforming baseline models on most tasks. It has potential applications in traffic analysis and modeling, such as generating synthetic traffic scenarios, explaining traffic phenomena, answering traffic-related questions, providing traffic recommendations, and generating traffic reports. The paper introduces TransGPT, two datasets (STD and MTD), and evaluates its performance on transportation benchmarks. The model is constructed using data collection, base model selection, and training processes. The results show that TransGPT achieves superior performance compared to other models. The paper also discusses related work on non-transformer and transformer-based models for transportation systems, highlighting the importance of domain-specific knowledge and data in transportation NLP. The study contributes to the advancement of NLP in the transportation domain and provides a useful tool for ITS researchers and practitioners.TransGPT is a novel multi-modal large language model designed for the transportation domain, consisting of two variants: TransGPT-SM for single-modal data and TransGPT-MM for multi-modal data. TransGPT-SM is fine-tuned on a single-modal transportation dataset (STD) containing 12.5 million tokens of textual data from various transportation sources. TransGPT-MM is fine-tuned on a multi-modal transportation dataset (MTD) manually collected from three areas: driving tests, traffic signs, and landmarks. The MTD includes aligned images and texts, with texts divided into questions and answers. TransGPT is evaluated on benchmark datasets for various transportation tasks, outperforming baseline models on most tasks. It has potential applications in traffic analysis and modeling, such as generating synthetic traffic scenarios, explaining traffic phenomena, answering traffic-related questions, providing traffic recommendations, and generating traffic reports. The paper introduces TransGPT, two datasets (STD and MTD), and evaluates its performance on transportation benchmarks. The model is constructed using data collection, base model selection, and training processes. The results show that TransGPT achieves superior performance compared to other models. The paper also discusses related work on non-transformer and transformer-based models for transportation systems, highlighting the importance of domain-specific knowledge and data in transportation NLP. The study contributes to the advancement of NLP in the transportation domain and provides a useful tool for ITS researchers and practitioners.
Reach us at info@study.space