TRUSTWORTHY CLASSIFICATION THROUGH RANK-BASED CONFORMAL PREDICTION SETS

TRUSTWORTHY CLASSIFICATION THROUGH RANK-BASED CONFORMAL PREDICTION SETS

5 Jul 2024 | Rui Luo* and Zhixin Zhou†
The paper "Trustworthy Classification through Rank-Based Conformal Prediction Sets" by Rui Luo and Zhixin Zhou proposes a novel conformal prediction method for classification tasks. The method, called RANK, addresses the challenge of constructing prediction sets with desired coverage while managing their size, even when the underlying classification model does not output well-calibrated probabilities. RANK uses a rank-based conformity score function that leverages the order of labels predicted by the model, rather than their calibrated probabilities. The authors provide a theoretical analysis of the expected size of the conformal prediction sets based on the rank distribution of the classifier. Extensive experiments on various datasets demonstrate that RANK outperforms existing techniques in terms of coverage and prediction set size, making it a reliable tool for uncertainty quantification in machine learning systems. The key contributions include a novel conformal prediction method, theoretical analysis, and empirical evaluation, advancing the practical deployment of machine learning systems.The paper "Trustworthy Classification through Rank-Based Conformal Prediction Sets" by Rui Luo and Zhixin Zhou proposes a novel conformal prediction method for classification tasks. The method, called RANK, addresses the challenge of constructing prediction sets with desired coverage while managing their size, even when the underlying classification model does not output well-calibrated probabilities. RANK uses a rank-based conformity score function that leverages the order of labels predicted by the model, rather than their calibrated probabilities. The authors provide a theoretical analysis of the expected size of the conformal prediction sets based on the rank distribution of the classifier. Extensive experiments on various datasets demonstrate that RANK outperforms existing techniques in terms of coverage and prediction set size, making it a reliable tool for uncertainty quantification in machine learning systems. The key contributions include a novel conformal prediction method, theoretical analysis, and empirical evaluation, advancing the practical deployment of machine learning systems.
Reach us at info@study.space