The paper introduces LabelMe, a web-based tool and database designed for image annotation and the collection of ground truth labels for object detection and recognition research. The authors aim to build a large dataset with diverse object categories, spanning various scenes and providing detailed information such as bounding boxes, polygons, and segmentation masks. LabelMe leverages the collaborative effort of a large user base to collect and share annotations, making it a valuable resource for supervised learning and quantitative evaluation in computer vision. The paper also discusses the limitations of existing datasets and highlights how LabelMe addresses these issues by offering high-quality, diverse, and context-rich annotations. Additionally, the authors present extensions and applications of the dataset, demonstrating its utility in enhancing object labels, discovering object parts, and recovering depth order in scenes.The paper introduces LabelMe, a web-based tool and database designed for image annotation and the collection of ground truth labels for object detection and recognition research. The authors aim to build a large dataset with diverse object categories, spanning various scenes and providing detailed information such as bounding boxes, polygons, and segmentation masks. LabelMe leverages the collaborative effort of a large user base to collect and share annotations, making it a valuable resource for supervised learning and quantitative evaluation in computer vision. The paper also discusses the limitations of existing datasets and highlights how LabelMe addresses these issues by offering high-quality, diverse, and context-rich annotations. Additionally, the authors present extensions and applications of the dataset, demonstrating its utility in enhancing object labels, discovering object parts, and recovering depth order in scenes.