Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty

Conformal Semantic Image Segmentation: Post-hoc Quantification of Predictive Uncertainty

16 Apr 2024 | Luca Mossina, Joseba Dalmau, Léo Andéol
This paper proposes a post-hoc method to quantify predictive uncertainty in semantic image segmentation using conformal prediction. The method generates statistically valid prediction sets that are guaranteed to include the ground-truth segmentation mask at a predefined confidence level. It introduces a novel visualization technique based on heatmaps to assess the empirical validity of these prediction sets. The approach is computationally lightweight and works with any segmentation model that outputs softmax scores, regardless of its architecture or training data distribution. The method quantifies uncertainty in the form of multi-labeled masks, where each pixel can have multiple labels. The coverage parameter λ determines the number of labels per pixel, with higher values leading to more labels. The user defines a risk level α and a minimum coverage ratio τ to select the optimal λ. The method ensures that the risk remains below the maximum tolerated level, providing a theoretical guarantee. The paper demonstrates the effectiveness of the approach on benchmark datasets and shows how the results can be visualized using varisco heatmaps. The method is compared to existing uncertainty quantification techniques and is shown to provide a complementary approach to calibration. The paper also discusses the characteristics of the heatmaps and their ability to visualize uncertainty in semantic segmentation tasks. The results show that the method provides valid uncertainty estimates and can be used to assess the performance of segmentation models. The paper concludes that the proposed method offers a computationally efficient way to quantify predictive uncertainty in semantic image segmentation with theoretical guarantees.This paper proposes a post-hoc method to quantify predictive uncertainty in semantic image segmentation using conformal prediction. The method generates statistically valid prediction sets that are guaranteed to include the ground-truth segmentation mask at a predefined confidence level. It introduces a novel visualization technique based on heatmaps to assess the empirical validity of these prediction sets. The approach is computationally lightweight and works with any segmentation model that outputs softmax scores, regardless of its architecture or training data distribution. The method quantifies uncertainty in the form of multi-labeled masks, where each pixel can have multiple labels. The coverage parameter λ determines the number of labels per pixel, with higher values leading to more labels. The user defines a risk level α and a minimum coverage ratio τ to select the optimal λ. The method ensures that the risk remains below the maximum tolerated level, providing a theoretical guarantee. The paper demonstrates the effectiveness of the approach on benchmark datasets and shows how the results can be visualized using varisco heatmaps. The method is compared to existing uncertainty quantification techniques and is shown to provide a complementary approach to calibration. The paper also discusses the characteristics of the heatmaps and their ability to visualize uncertainty in semantic segmentation tasks. The results show that the method provides valid uncertainty estimates and can be used to assess the performance of segmentation models. The paper concludes that the proposed method offers a computationally efficient way to quantify predictive uncertainty in semantic image segmentation with theoretical guarantees.
Reach us at info@study.space