This paper presents a comprehensive study and evaluation of existing single image dehazing algorithms using a new large-scale benchmark called REAlistic Single Image DEhazing (RESIDE). RESIDE consists of synthetic and real-world hazy images, divided into five subsets for different training and evaluation purposes. The paper introduces a variety of evaluation criteria, including full-reference metrics, no-reference metrics, subjective evaluation, and task-driven evaluation. The experiments on RESIDE reveal the performance and limitations of state-of-the-art dehazing algorithms and suggest future research directions. The RESIDE dataset is publicly available for research purposes, and the paper also discusses the challenges and limitations of current dehazing algorithms, such as the mismatch between training data and real-world applications and the lack of appropriate evaluation criteria. The paper concludes with a discussion on the complex nature of the dehazing problem and suggests future research directions, including evaluating algorithms with more dedicated criteria and developing no-reference metrics that better reflect human perception.This paper presents a comprehensive study and evaluation of existing single image dehazing algorithms using a new large-scale benchmark called REAlistic Single Image DEhazing (RESIDE). RESIDE consists of synthetic and real-world hazy images, divided into five subsets for different training and evaluation purposes. The paper introduces a variety of evaluation criteria, including full-reference metrics, no-reference metrics, subjective evaluation, and task-driven evaluation. The experiments on RESIDE reveal the performance and limitations of state-of-the-art dehazing algorithms and suggest future research directions. The RESIDE dataset is publicly available for research purposes, and the paper also discusses the challenges and limitations of current dehazing algorithms, such as the mismatch between training data and real-world applications and the lack of appropriate evaluation criteria. The paper concludes with a discussion on the complex nature of the dehazing problem and suggests future research directions, including evaluating algorithms with more dedicated criteria and developing no-reference metrics that better reflect human perception.