UnO: Unsupervised Occupancy Fields for Perception and Forecasting

UnO: Unsupervised Occupancy Fields for Perception and Forecasting

12 Jun 2024 | Ben Agro, Quinlan Sykora, Sergio Casas, Thomas Gilles, Raquel Urtasun
UNO is an unsupervised world model that learns to predict 3D occupancy over time from unlabeled data. It can be effectively transferred to downstream tasks such as point cloud forecasting and BEV semantic occupancy prediction. The model learns a continuous 4D occupancy field from LiDAR data, enabling it to understand the geometry, dynamics, and semantics of the environment. UNO outperforms state-of-the-art methods in point cloud forecasting on Argoverse 2, nuScenes, and KITTI, and achieves better performance in BEV semantic occupancy prediction even with limited labeled data. The model's implicit architecture allows it to be queried at any continuous point in space and time, enabling accurate predictions of future states. UNO's ability to forecast multi-modal futures with associated uncertainty is demonstrated through its predictions of vehicle lane changes and pedestrian movements. The model is also effective in predicting occupancy for rare and small objects, showing strong performance in geometric occupancy forecasting. UNO's unsupervised learning approach allows it to generalize well to various downstream tasks, making it a valuable tool for self-driving systems.UNO is an unsupervised world model that learns to predict 3D occupancy over time from unlabeled data. It can be effectively transferred to downstream tasks such as point cloud forecasting and BEV semantic occupancy prediction. The model learns a continuous 4D occupancy field from LiDAR data, enabling it to understand the geometry, dynamics, and semantics of the environment. UNO outperforms state-of-the-art methods in point cloud forecasting on Argoverse 2, nuScenes, and KITTI, and achieves better performance in BEV semantic occupancy prediction even with limited labeled data. The model's implicit architecture allows it to be queried at any continuous point in space and time, enabling accurate predictions of future states. UNO's ability to forecast multi-modal futures with associated uncertainty is demonstrated through its predictions of vehicle lane changes and pedestrian movements. The model is also effective in predicting occupancy for rare and small objects, showing strong performance in geometric occupancy forecasting. UNO's unsupervised learning approach allows it to generalize well to various downstream tasks, making it a valuable tool for self-driving systems.
Reach us at info@study.space
[slides] UnO%3A Unsupervised Occupancy Fields for Perception and Forecasting | StudySpace