The paper introduces the Caltech Pedestrian Dataset, a large-scale dataset for pedestrian detection, which is two orders of magnitude larger than existing datasets. The dataset includes richly annotated video from a moving vehicle, capturing challenging images of low resolution and frequently occluded people. The authors propose improved evaluation metrics, demonstrating that commonly used per-window measures are flawed and can fail to predict performance on full images. They benchmark several promising detection systems, providing an overview of state-of-the-art performance and a direct, unbiased comparison of existing methods. By analyzing common failure cases, they help identify future research directions for the field. The dataset and evaluation methodology are designed to address the limitations of existing benchmarks and to inspire novel ideas in pedestrian detection.The paper introduces the Caltech Pedestrian Dataset, a large-scale dataset for pedestrian detection, which is two orders of magnitude larger than existing datasets. The dataset includes richly annotated video from a moving vehicle, capturing challenging images of low resolution and frequently occluded people. The authors propose improved evaluation metrics, demonstrating that commonly used per-window measures are flawed and can fail to predict performance on full images. They benchmark several promising detection systems, providing an overview of state-of-the-art performance and a direct, unbiased comparison of existing methods. By analyzing common failure cases, they help identify future research directions for the field. The dataset and evaluation methodology are designed to address the limitations of existing benchmarks and to inspire novel ideas in pedestrian detection.