The paper introduces a new task called Panoptic Segmentation (PS), which unifies the tasks of semantic segmentation and instance segmentation. PS aims to generate a coherent and rich scene segmentation, addressing both stuff (amorphous regions) and thing (countable objects) classes. The authors propose a novel metric, Panoptic Quality (PQ), to evaluate performance for all classes in a unified manner. They perform a detailed study of human and machine performance on three datasets (Cityscapes, ADE20k, and Mapillary Vistas), revealing insights into the task's challenges and potential. The paper also discusses related work, including object detection, semantic segmentation, multitask learning, and joint segmentation tasks. The authors hope that the introduction of PS will revive interest in a more unified view of image segmentation and drive innovation in the field.The paper introduces a new task called Panoptic Segmentation (PS), which unifies the tasks of semantic segmentation and instance segmentation. PS aims to generate a coherent and rich scene segmentation, addressing both stuff (amorphous regions) and thing (countable objects) classes. The authors propose a novel metric, Panoptic Quality (PQ), to evaluate performance for all classes in a unified manner. They perform a detailed study of human and machine performance on three datasets (Cityscapes, ADE20k, and Mapillary Vistas), revealing insights into the task's challenges and potential. The paper also discusses related work, including object detection, semantic segmentation, multitask learning, and joint segmentation tasks. The authors hope that the introduction of PS will revive interest in a more unified view of image segmentation and drive innovation in the field.