TRUSTWORTHY CLASSIFICATION THROUGH RANK-BASED CONFORMAL PREDICTION SETS

TRUSTWORTHY CLASSIFICATION THROUGH RANK-BASED CONFORMAL PREDICTION SETS

5 Jul 2024 | Rui Luo and Zhixin Zhou
This paper proposes a novel conformal prediction method called RANK for classification tasks, which uses a rank-based score function to construct prediction sets with desired coverage and minimal size. The method is suitable for classification models that may not output well-calibrated probabilities but can correctly rank labels. The approach constructs prediction sets that achieve the desired coverage rate while managing their size. Theoretical analysis is provided to evaluate the expected size of the conformal prediction sets based on the rank distribution of the underlying classifier. Extensive experiments on various datasets demonstrate that the proposed method outperforms existing techniques in terms of coverage and prediction set size. The key contributions include a novel conformal prediction method, theoretical analysis, and empirical evaluation. The work advances the practical deployment of machine learning systems by enabling reliable uncertainty quantification.This paper proposes a novel conformal prediction method called RANK for classification tasks, which uses a rank-based score function to construct prediction sets with desired coverage and minimal size. The method is suitable for classification models that may not output well-calibrated probabilities but can correctly rank labels. The approach constructs prediction sets that achieve the desired coverage rate while managing their size. Theoretical analysis is provided to evaluate the expected size of the conformal prediction sets based on the rank distribution of the underlying classifier. Extensive experiments on various datasets demonstrate that the proposed method outperforms existing techniques in terms of coverage and prediction set size. The key contributions include a novel conformal prediction method, theoretical analysis, and empirical evaluation. The work advances the practical deployment of machine learning systems by enabling reliable uncertainty quantification.
Reach us at info@study.space