The paper introduces nuScenes, a comprehensive multimodal dataset for autonomous driving research. nuScenes is the first dataset to capture the full suite of autonomous vehicle sensors, including 6 cameras, 5 radars, and 1 lidar, all with a 360-degree field of view. The dataset comprises 1000 scenes, each 20 seconds long, fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It provides 7 times more annotations and 100 times more images compared to the KITTI dataset. The authors define new 3D detection and tracking metrics and provide baselines for lidar and image-based methods. The dataset includes nighttime and rainy conditions, semantic maps, and object attributes, making it suitable for a wide range of tasks such as object detection, tracking, and behavior modeling. The paper also discusses the importance of large-scale datasets and the impact of sensor fusion on performance, highlighting the benefits of using both lidar and image data. The release of nuScenes has sparked significant interest in the AV community, leading to advancements in detection and tracking methods.The paper introduces nuScenes, a comprehensive multimodal dataset for autonomous driving research. nuScenes is the first dataset to capture the full suite of autonomous vehicle sensors, including 6 cameras, 5 radars, and 1 lidar, all with a 360-degree field of view. The dataset comprises 1000 scenes, each 20 seconds long, fully annotated with 3D bounding boxes for 23 classes and 8 attributes. It provides 7 times more annotations and 100 times more images compared to the KITTI dataset. The authors define new 3D detection and tracking metrics and provide baselines for lidar and image-based methods. The dataset includes nighttime and rainy conditions, semantic maps, and object attributes, making it suitable for a wide range of tasks such as object detection, tracking, and behavior modeling. The paper also discusses the importance of large-scale datasets and the impact of sensor fusion on performance, highlighting the benefits of using both lidar and image data. The release of nuScenes has sparked significant interest in the AV community, leading to advancements in detection and tracking methods.