CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks

CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks

2 Feb 2024 | Kaizheng Wang, Keivan Shariatmadar, Shireen Kudukkil Manchingal, Fabio Cuzzolin, David Moens, Hans Hallez
CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks This paper introduces CreINNs, a novel credal-set interval neural network for uncertainty estimation in classification tasks. CreINNs retain the traditional interval neural network structure, capturing weight uncertainty through deterministic intervals, while forecasting credal sets using the mathematical framework of probability intervals. Experimental results on an out-of-distribution detection benchmark (CIFAR10 vs SVHN) show that CreINNs outperform variational Bayesian neural networks (BNNs) and deep ensembles (DEs) in epistemic uncertainty estimation. Additionally, CreINNs demonstrate a notable reduction in computational complexity compared to BNNs and smaller model sizes than DEs. CreINNs are designed to transform output interval scores into probability intervals while satisfying the condition that the resulting probability bounds for each class strictly adhere to the lower and upper bounds. This is achieved through an original Interval Softmax activation function. CreINNs also incorporate Interval Batch Normalization to improve training stability and adaptability to large and deep network architectures. The paper presents a detailed overview of CreINNs, including their structure, training procedure, and implementation of Interval Softmax and Interval Batch Normalization. Experimental validation on CIFAR10 vs SVHN datasets shows that CreINNs outperform BNNs and DEs in terms of inference time and OoD detection performance. The results indicate that CreINNs are particularly effective in estimating epistemic uncertainty, as evidenced by their superior AUROC and AUPRC scores. The paper also includes an ablation study on the hyperparameter δ, showing that varying δ values affect test accuracy, AUROC, and AUPRC scores. The results confirm the robustness of CreINNs' uncertainty estimation across different hyperparameter settings. In conclusion, CreINNs offer a promising approach for uncertainty estimation in classification tasks, with superior performance in terms of epistemic uncertainty estimation, computational efficiency, and model size. Future work will focus on reducing computational complexity and developing a comprehensive evaluation framework for various uncertainty-aware models.CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks This paper introduces CreINNs, a novel credal-set interval neural network for uncertainty estimation in classification tasks. CreINNs retain the traditional interval neural network structure, capturing weight uncertainty through deterministic intervals, while forecasting credal sets using the mathematical framework of probability intervals. Experimental results on an out-of-distribution detection benchmark (CIFAR10 vs SVHN) show that CreINNs outperform variational Bayesian neural networks (BNNs) and deep ensembles (DEs) in epistemic uncertainty estimation. Additionally, CreINNs demonstrate a notable reduction in computational complexity compared to BNNs and smaller model sizes than DEs. CreINNs are designed to transform output interval scores into probability intervals while satisfying the condition that the resulting probability bounds for each class strictly adhere to the lower and upper bounds. This is achieved through an original Interval Softmax activation function. CreINNs also incorporate Interval Batch Normalization to improve training stability and adaptability to large and deep network architectures. The paper presents a detailed overview of CreINNs, including their structure, training procedure, and implementation of Interval Softmax and Interval Batch Normalization. Experimental validation on CIFAR10 vs SVHN datasets shows that CreINNs outperform BNNs and DEs in terms of inference time and OoD detection performance. The results indicate that CreINNs are particularly effective in estimating epistemic uncertainty, as evidenced by their superior AUROC and AUPRC scores. The paper also includes an ablation study on the hyperparameter δ, showing that varying δ values affect test accuracy, AUROC, and AUPRC scores. The results confirm the robustness of CreINNs' uncertainty estimation across different hyperparameter settings. In conclusion, CreINNs offer a promising approach for uncertainty estimation in classification tasks, with superior performance in terms of epistemic uncertainty estimation, computational efficiency, and model size. Future work will focus on reducing computational complexity and developing a comprehensive evaluation framework for various uncertainty-aware models.
Reach us at info@study.space
[slides and audio] CreINNs%3A Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks