12 Apr 2024 | Etash Guha, Shlok Natarajan, Thomas Möllenhoff, Mohammad Emtiyaz Khan, Eugene Ndiaye
The paper presents a novel approach to Conformal Prediction (CP) for regression, which is particularly challenging when the output distribution is heteroscedastic, multimodal, or skewed. The authors propose converting the regression problem into a classification problem and then applying CP techniques for classification to obtain conformal sets for regression. This approach preserves the ordering of the continuous output space and addresses the limitations of traditional CP methods, which can be sensitive to estimation errors and yield unstable intervals.
To achieve this, the authors design a new loss function that penalizes the density on bins far from the true output value while facilitating variability through entropy regularization. This loss function is similar to those used in classification conformal prediction and allows the method to adapt to various distribution shapes, including heteroscedasticity and bimodality.
The empirical results on synthetic and real datasets demonstrate that the proposed method achieves shorter intervals compared to other CP baselines, showing its effectiveness in handling complex output distributions. The authors also conduct ablation studies to validate the importance of the proposed loss function components and discuss the limitations of their approach, such as the need for larger neural networks and hyperparameter tuning for different datasets.The paper presents a novel approach to Conformal Prediction (CP) for regression, which is particularly challenging when the output distribution is heteroscedastic, multimodal, or skewed. The authors propose converting the regression problem into a classification problem and then applying CP techniques for classification to obtain conformal sets for regression. This approach preserves the ordering of the continuous output space and addresses the limitations of traditional CP methods, which can be sensitive to estimation errors and yield unstable intervals.
To achieve this, the authors design a new loss function that penalizes the density on bins far from the true output value while facilitating variability through entropy regularization. This loss function is similar to those used in classification conformal prediction and allows the method to adapt to various distribution shapes, including heteroscedasticity and bimodality.
The empirical results on synthetic and real datasets demonstrate that the proposed method achieves shorter intervals compared to other CP baselines, showing its effectiveness in handling complex output distributions. The authors also conduct ablation studies to validate the importance of the proposed loss function components and discuss the limitations of their approach, such as the need for larger neural networks and hyperparameter tuning for different datasets.