CARLA: An Open Urban Driving Simulator

CARLA: An Open Urban Driving Simulator

10 Nov 2017 | Alexey Dosovitskiy, German Ros, Felipe Codevilla, Antonio López, and Vladlen Koltun
CARLA is an open-source simulator for autonomous driving research, developed to support the development, training, and validation of autonomous urban driving systems. It provides open-source code, protocols, and digital assets such as urban layouts, buildings, and vehicles. The simulation platform supports flexible sensor configurations and environmental conditions. CARLA is used to evaluate three autonomous driving approaches: a classic modular pipeline, an end-to-end model trained via imitation learning, and an end-to-end model trained via reinforcement learning. These approaches are tested in controlled scenarios of increasing difficulty, with performance evaluated using metrics provided by CARLA, demonstrating the platform's utility for autonomous driving research. The simulator is built on Unreal Engine 4 and offers state-of-the-art rendering quality, realistic physics, and an ecosystem of interoperable plugins. It supports flexible setup of sensor suites and provides signals for training driving strategies, such as GPS coordinates, speed, acceleration, and collision data. A wide range of environmental conditions can be specified, including weather and time of day. CARLA simulates a dynamic world with 3D models of static and dynamic objects, including buildings, vehicles, and pedestrians. The environment includes two towns with different drivable road lengths for training and testing. CARLA supports the development and evaluation of autonomous driving systems. Three approaches are tested: a modular pipeline with perception, planning, and control modules; a deep network trained via imitation learning; and a deep network trained via reinforcement learning. The modular pipeline uses a semantic segmentation network for perception, a rule-based state machine for planning, and a PID controller for continuous control. The imitation learning approach uses a dataset of human driving traces to train a deep network. The reinforcement learning approach uses the A3C algorithm to train a deep network based on reward signals. Experiments show that all methods perform well on simple tasks but struggle with more complex ones. Generalization to new weather is easier than generalization to a new town. The modular pipeline and imitation learning perform similarly, while reinforcement learning underperforms. Results highlight the importance of generalization for learning-based approaches to sensorimotor control. The modular pipeline is more fragile than end-to-end methods. Imitation learning performs better than reinforcement learning on most tasks, despite the latter being trained on more data. Results also show that reinforcement learning has poor generalization, possibly due to lack of data augmentation. CARLA provides tools for detailed analysis of driving policies, highlighting failure modes and opportunities for future work. The simulator and its assets are released open-source for the broader autonomous driving research community.CARLA is an open-source simulator for autonomous driving research, developed to support the development, training, and validation of autonomous urban driving systems. It provides open-source code, protocols, and digital assets such as urban layouts, buildings, and vehicles. The simulation platform supports flexible sensor configurations and environmental conditions. CARLA is used to evaluate three autonomous driving approaches: a classic modular pipeline, an end-to-end model trained via imitation learning, and an end-to-end model trained via reinforcement learning. These approaches are tested in controlled scenarios of increasing difficulty, with performance evaluated using metrics provided by CARLA, demonstrating the platform's utility for autonomous driving research. The simulator is built on Unreal Engine 4 and offers state-of-the-art rendering quality, realistic physics, and an ecosystem of interoperable plugins. It supports flexible setup of sensor suites and provides signals for training driving strategies, such as GPS coordinates, speed, acceleration, and collision data. A wide range of environmental conditions can be specified, including weather and time of day. CARLA simulates a dynamic world with 3D models of static and dynamic objects, including buildings, vehicles, and pedestrians. The environment includes two towns with different drivable road lengths for training and testing. CARLA supports the development and evaluation of autonomous driving systems. Three approaches are tested: a modular pipeline with perception, planning, and control modules; a deep network trained via imitation learning; and a deep network trained via reinforcement learning. The modular pipeline uses a semantic segmentation network for perception, a rule-based state machine for planning, and a PID controller for continuous control. The imitation learning approach uses a dataset of human driving traces to train a deep network. The reinforcement learning approach uses the A3C algorithm to train a deep network based on reward signals. Experiments show that all methods perform well on simple tasks but struggle with more complex ones. Generalization to new weather is easier than generalization to a new town. The modular pipeline and imitation learning perform similarly, while reinforcement learning underperforms. Results highlight the importance of generalization for learning-based approaches to sensorimotor control. The modular pipeline is more fragile than end-to-end methods. Imitation learning performs better than reinforcement learning on most tasks, despite the latter being trained on more data. Results also show that reinforcement learning has poor generalization, possibly due to lack of data augmentation. CARLA provides tools for detailed analysis of driving policies, highlighting failure modes and opportunities for future work. The simulator and its assets are released open-source for the broader autonomous driving research community.
Reach us at info@study.space
[slides and audio] CARLA%3A An Open Urban Driving Simulator