LabelMe is a database and web-based tool for image annotation, designed to collect a large collection of images with ground truth labels for object detection and recognition research. The tool allows easy image annotation and instant sharing of annotations. Using this tool, a large dataset has been collected spanning many object categories, often containing multiple instances across various images. The dataset's contents are quantified and compared with existing state-of-the-art datasets for object recognition and detection. The dataset can be extended to automatically enhance object labels with WordNet, discover object parts, recover depth ordering of objects in a scene, and increase the number of labels with minimal user supervision and web images. The paper discusses the importance of labeled data for supervised learning and quantitative evaluation of object detection and recognition algorithms. It highlights the limitations of current methods in scaling to thousands of object categories and the need for large image and video collections with ground truth labels spanning many object categories in cluttered scenes. The paper also discusses the benefits of using labeled data for comparing different algorithms and the challenges of building large annotated image datasets. LabelMe aims to collect a large dataset of annotated images by leveraging web-based data collection methods. Web-based annotation tools allow the collaborative effort of a large population of users to build large annotated datasets. Examples include the Open Mind Initiative, which collects common sense facts, and the ESP game, which collects image captions. However, these efforts often only collect caption data, not detailed object labels. The paper describes LabelMe, a database and online annotation tool that allows sharing of images and annotations. The tool provides functionalities such as drawing polygons, querying images, and browsing the database. The paper evaluates the quality of the labeling and presents extensions and applications of the dataset. It also compares the LabelMe dataset against other existing datasets commonly used for object detection and recognition. The LabelMe dataset is designed for object class recognition, learning about objects embedded in a scene, high-quality labeling, and many diverse object classes.LabelMe is a database and web-based tool for image annotation, designed to collect a large collection of images with ground truth labels for object detection and recognition research. The tool allows easy image annotation and instant sharing of annotations. Using this tool, a large dataset has been collected spanning many object categories, often containing multiple instances across various images. The dataset's contents are quantified and compared with existing state-of-the-art datasets for object recognition and detection. The dataset can be extended to automatically enhance object labels with WordNet, discover object parts, recover depth ordering of objects in a scene, and increase the number of labels with minimal user supervision and web images. The paper discusses the importance of labeled data for supervised learning and quantitative evaluation of object detection and recognition algorithms. It highlights the limitations of current methods in scaling to thousands of object categories and the need for large image and video collections with ground truth labels spanning many object categories in cluttered scenes. The paper also discusses the benefits of using labeled data for comparing different algorithms and the challenges of building large annotated image datasets. LabelMe aims to collect a large dataset of annotated images by leveraging web-based data collection methods. Web-based annotation tools allow the collaborative effort of a large population of users to build large annotated datasets. Examples include the Open Mind Initiative, which collects common sense facts, and the ESP game, which collects image captions. However, these efforts often only collect caption data, not detailed object labels. The paper describes LabelMe, a database and online annotation tool that allows sharing of images and annotations. The tool provides functionalities such as drawing polygons, querying images, and browsing the database. The paper evaluates the quality of the labeling and presents extensions and applications of the dataset. It also compares the LabelMe dataset against other existing datasets commonly used for object detection and recognition. The LabelMe dataset is designed for object class recognition, learning about objects embedded in a scene, high-quality labeling, and many diverse object classes.