This paper presents a comprehensive study and evaluation of existing single image dehazing algorithms using a new large-scale benchmark called RESIDE, which includes both synthetic and real-world hazy images. RESIDE is divided into five subsets for different training and evaluation purposes and provides a variety of evaluation criteria, including full-reference, no-reference, and subjective evaluations. The benchmark highlights the diversity of data sources and image contents, and includes a task-driven evaluation set of 4,322 real-world hazy images with object bounding boxes. The paper discusses the problem of single image dehazing, which is challenging due to the presence of haze, which adds complicated, nonlinear and data-dependent noise to images. The atmospheric scattering model is used to describe the generation of hazy images, and most state-of-the-art dehazing methods exploit this model to estimate key parameters. The paper also reviews existing methodologies, including the use of natural image priors, depth statistics, and CNN-based approaches. The paper introduces the RESIDE dataset, which includes a large-scale synthetic training set and two different sets for objective and subjective quality evaluations. It also introduces the RESIDE-β set, which includes innovative discussions on training data content and evaluation criteria. The paper evaluates nine state-of-the-art single image dehazing algorithms using the RESIDE and RESIDE-β datasets and various evaluation criteria. The results show that learning-based methods outperform earlier algorithms based on natural or statistical priors in terms of PSNR and SSIM. However, no-reference metrics show less consistency, with some prior-based methods also showing competitiveness. The paper also discusses the challenges of using synthetic data for training dehazing models, and proposes the use of real-world outdoor images for training. It also explores the use of task-driven evaluation methods, such as object detection performance, to evaluate dehazing results. The paper concludes that dehazing is an increasingly important technique for both computational photography and computer vision tasks, and that future research should focus on more dedicated criteria, such as subjective visual quality or high-level target task performance, rather than solely PSNR/SSIM. The paper also highlights the need for developing no-reference metrics that are better correlated with human perception for evaluating dehazing results.This paper presents a comprehensive study and evaluation of existing single image dehazing algorithms using a new large-scale benchmark called RESIDE, which includes both synthetic and real-world hazy images. RESIDE is divided into five subsets for different training and evaluation purposes and provides a variety of evaluation criteria, including full-reference, no-reference, and subjective evaluations. The benchmark highlights the diversity of data sources and image contents, and includes a task-driven evaluation set of 4,322 real-world hazy images with object bounding boxes. The paper discusses the problem of single image dehazing, which is challenging due to the presence of haze, which adds complicated, nonlinear and data-dependent noise to images. The atmospheric scattering model is used to describe the generation of hazy images, and most state-of-the-art dehazing methods exploit this model to estimate key parameters. The paper also reviews existing methodologies, including the use of natural image priors, depth statistics, and CNN-based approaches. The paper introduces the RESIDE dataset, which includes a large-scale synthetic training set and two different sets for objective and subjective quality evaluations. It also introduces the RESIDE-β set, which includes innovative discussions on training data content and evaluation criteria. The paper evaluates nine state-of-the-art single image dehazing algorithms using the RESIDE and RESIDE-β datasets and various evaluation criteria. The results show that learning-based methods outperform earlier algorithms based on natural or statistical priors in terms of PSNR and SSIM. However, no-reference metrics show less consistency, with some prior-based methods also showing competitiveness. The paper also discusses the challenges of using synthetic data for training dehazing models, and proposes the use of real-world outdoor images for training. It also explores the use of task-driven evaluation methods, such as object detection performance, to evaluate dehazing results. The paper concludes that dehazing is an increasingly important technique for both computational photography and computer vision tasks, and that future research should focus on more dedicated criteria, such as subjective visual quality or high-level target task performance, rather than solely PSNR/SSIM. The paper also highlights the need for developing no-reference metrics that are better correlated with human perception for evaluating dehazing results.