The SUN database is a large-scale scene recognition dataset containing 899 categories and 130,519 images, designed to address the limitations of existing scene databases that cover only a limited number of scene categories. The paper evaluates state-of-the-art scene recognition algorithms on this extensive dataset and compares their performance with human scene classification. It also explores finer-grained scene representation and scene detection within images. The SUN database was created by selecting terms from WordNet that describe scenes, places, and environments, and then refining them to ensure they represent distinct and meaningful scene categories. The dataset includes a wide variety of scenes, such as abbeys, churches, zoos, and more, and was curated to cover as many different scene types as possible. The images were collected from the web, and each image was carefully selected to fit the defined categories. Human scene classification performance was measured using Amazon's Mechanical Turk, where participants were asked to classify images into the appropriate scene categories. The results showed that humans achieved a high level of accuracy, with an average of 58.6% accuracy at the leaf level. However, some categories were more challenging for humans, with accuracy as low as 49.6%. The paper also evaluates the performance of computational methods on the SUN database. The results show that while computational methods can achieve high accuracy on some categories, they often struggle with categories that are semantically similar but visually distinct. The best computational performance was achieved on outdoor natural scenes, while the worst was on outdoor man-made scenes. The paper also introduces the concept of scene detection, which involves identifying scene types within image regions rather than entire images. This is a new task that builds on existing object detection techniques. The paper presents a method for scene detection that uses a multiscale scanning-window approach to find sub-scenes within images. The SUN database and the associated experiments provide a comprehensive evaluation of scene recognition and detection, highlighting the challenges and opportunities in this area of computer vision. The results suggest that while computational methods can achieve high accuracy on some categories, they still have limitations in capturing the complexity and diversity of real-world scenes. The SUN database is a valuable resource for researchers in the field of scene understanding and recognition.The SUN database is a large-scale scene recognition dataset containing 899 categories and 130,519 images, designed to address the limitations of existing scene databases that cover only a limited number of scene categories. The paper evaluates state-of-the-art scene recognition algorithms on this extensive dataset and compares their performance with human scene classification. It also explores finer-grained scene representation and scene detection within images. The SUN database was created by selecting terms from WordNet that describe scenes, places, and environments, and then refining them to ensure they represent distinct and meaningful scene categories. The dataset includes a wide variety of scenes, such as abbeys, churches, zoos, and more, and was curated to cover as many different scene types as possible. The images were collected from the web, and each image was carefully selected to fit the defined categories. Human scene classification performance was measured using Amazon's Mechanical Turk, where participants were asked to classify images into the appropriate scene categories. The results showed that humans achieved a high level of accuracy, with an average of 58.6% accuracy at the leaf level. However, some categories were more challenging for humans, with accuracy as low as 49.6%. The paper also evaluates the performance of computational methods on the SUN database. The results show that while computational methods can achieve high accuracy on some categories, they often struggle with categories that are semantically similar but visually distinct. The best computational performance was achieved on outdoor natural scenes, while the worst was on outdoor man-made scenes. The paper also introduces the concept of scene detection, which involves identifying scene types within image regions rather than entire images. This is a new task that builds on existing object detection techniques. The paper presents a method for scene detection that uses a multiscale scanning-window approach to find sub-scenes within images. The SUN database and the associated experiments provide a comprehensive evaluation of scene recognition and detection, highlighting the challenges and opportunities in this area of computer vision. The results suggest that while computational methods can achieve high accuracy on some categories, they still have limitations in capturing the complexity and diversity of real-world scenes. The SUN database is a valuable resource for researchers in the field of scene understanding and recognition.