17 May 2019 | Christos Sakaridis · Dengxin Dai · Luc Van Gool
This paper addresses the problem of semantic foggy scene understanding (SFSU), which has received less attention compared to image dehazing and semantic scene understanding with clear-weather images. To overcome the challenge of collecting and annotating foggy images, the authors generate synthetic fog on real images depicting clear-weather outdoor scenes and use these partially synthetic data for SFSU. They develop a complete pipeline to add synthetic fog to real, clear-weather images using incomplete depth information. The pipeline is applied to the Cityscapes dataset, resulting in 20,550 *Foggy Cityscapes* images. SFSU is tackled using two approaches: supervised learning and a novel semi-supervised learning method that combines unsupervised supervision transfer from clear-weather images to their synthetic foggy counterparts. The authors also study the usefulness of image dehazing for SFSU. For evaluation, they present *Foggy Driving*, a dataset with 101 real-world images of foggy driving scenes, featuring ground truth annotations for semantic segmentation and object detection. Extensive experiments show that supervised learning with synthetic data significantly improves the performance of state-of-the-art CNNs for SFSU on *Foggy Driving*, and the semi-supervised learning strategy further enhances performance. Image dehazing marginally improves SFSU with the proposed learning strategy. The datasets, models, and code are made publicly available.This paper addresses the problem of semantic foggy scene understanding (SFSU), which has received less attention compared to image dehazing and semantic scene understanding with clear-weather images. To overcome the challenge of collecting and annotating foggy images, the authors generate synthetic fog on real images depicting clear-weather outdoor scenes and use these partially synthetic data for SFSU. They develop a complete pipeline to add synthetic fog to real, clear-weather images using incomplete depth information. The pipeline is applied to the Cityscapes dataset, resulting in 20,550 *Foggy Cityscapes* images. SFSU is tackled using two approaches: supervised learning and a novel semi-supervised learning method that combines unsupervised supervision transfer from clear-weather images to their synthetic foggy counterparts. The authors also study the usefulness of image dehazing for SFSU. For evaluation, they present *Foggy Driving*, a dataset with 101 real-world images of foggy driving scenes, featuring ground truth annotations for semantic segmentation and object detection. Extensive experiments show that supervised learning with synthetic data significantly improves the performance of state-of-the-art CNNs for SFSU on *Foggy Driving*, and the semi-supervised learning strategy further enhances performance. Image dehazing marginally improves SFSU with the proposed learning strategy. The datasets, models, and code are made publicly available.