Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty

Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty

16 Apr 2024 | Luca Mossina, Joseba Dalmau, Léo Andéol
The paper introduces a post-hoc, computationally lightweight method to quantify predictive uncertainty in semantic image segmentation using conformal prediction. The approach generates statistically valid prediction sets that are guaranteed to include the ground-truth segmentation mask at a predefined confidence level. A novel visualization technique, called varisco heatmaps, is introduced to assess the empirical validity of these predictions. The method is demonstrated on well-known benchmark datasets and image segmentation prediction models, showing its effectiveness and practical insights. The contributions include a method based on Conformal Prediction (CP) to assess predictive uncertainty in pre-trained segmentation predictors, regardless of their architecture or training data distribution. The method quantifies uncertainty in the form of multi-labeled masks, which can take multiple labels per pixel, and provides theoretical guarantees through Conformal Risk Control (CRC). The paper also discusses related work, including uncertainty quantification methods for semantic image segmentation and applications of CP to segmentation. Experiments on various datasets and models, such as Cityscapes, ADE20K, and LoveDA, demonstrate the method's effectiveness and practicality.The paper introduces a post-hoc, computationally lightweight method to quantify predictive uncertainty in semantic image segmentation using conformal prediction. The approach generates statistically valid prediction sets that are guaranteed to include the ground-truth segmentation mask at a predefined confidence level. A novel visualization technique, called varisco heatmaps, is introduced to assess the empirical validity of these predictions. The method is demonstrated on well-known benchmark datasets and image segmentation prediction models, showing its effectiveness and practical insights. The contributions include a method based on Conformal Prediction (CP) to assess predictive uncertainty in pre-trained segmentation predictors, regardless of their architecture or training data distribution. The method quantifies uncertainty in the form of multi-labeled masks, which can take multiple labels per pixel, and provides theoretical guarantees through Conformal Risk Control (CRC). The paper also discusses related work, including uncertainty quantification methods for semantic image segmentation and applications of CP to segmentation. Experiments on various datasets and models, such as Cityscapes, ADE20K, and LoveDA, demonstrate the method's effectiveness and practicality.
Reach us at info@study.space
[slides] Conformal Semantic Image Segmentation%3A Post-hoc Quantification of Predictive Uncertainty | StudySpace