CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks

CreINNs: Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks

2 Feb 2024 | Kaizheng Wang, Keivan Shariatmadar, Shireen Kudukkil Manchingal, Fabio Cuzzolin, David Moens, Hans Hallez
The paper introduces a novel approach called Credal-Set Interval Neural Networks (CreINNs) for uncertainty estimation in classification tasks. CreINNs retain the structure of traditional interval neural networks (INNs) but use deterministic intervals to capture weight uncertainty and produce credal sets as predictions. The main contributions include: 1. **Interval Softmax Activation Function**: A novel activation function that converts interval-formed outputs into convex probability intervals, representing the lower and upper bounds of probabilities across classes. 2. **Credal Set Predictions**: Formulating credal set predictions in deep neural networks using probability intervals, allowing for the quantification and differentiation of aleatoric (AU) and epistemic (EU) uncertainties. 3. **Training Procedure**: Introducing a new training procedure that enables effective learning of deterministic parameter intervals and their associated credal sets. 4. **Interval Batch Normalization**: A heuristic approach to improve the stability of the training process and facilitate the adaptability of CreINNs to large and deep modern network architectures. Experimental results on an out-of-distribution (OoD) detection benchmark (CIFAR10 vs SVHN) using ResNet50 architecture show that CreINNs outperform variational Bayesian neural networks (BNNs) and deep ensembles (DEs) in terms of EU estimation. CreINNs also exhibit significant computational complexity reduction and smaller model sizes compared to BNNs and DEs. The paper concludes with discussions on future work, including reducing computational complexity and developing a comprehensive evaluation framework for uncertainty-aware models.The paper introduces a novel approach called Credal-Set Interval Neural Networks (CreINNs) for uncertainty estimation in classification tasks. CreINNs retain the structure of traditional interval neural networks (INNs) but use deterministic intervals to capture weight uncertainty and produce credal sets as predictions. The main contributions include: 1. **Interval Softmax Activation Function**: A novel activation function that converts interval-formed outputs into convex probability intervals, representing the lower and upper bounds of probabilities across classes. 2. **Credal Set Predictions**: Formulating credal set predictions in deep neural networks using probability intervals, allowing for the quantification and differentiation of aleatoric (AU) and epistemic (EU) uncertainties. 3. **Training Procedure**: Introducing a new training procedure that enables effective learning of deterministic parameter intervals and their associated credal sets. 4. **Interval Batch Normalization**: A heuristic approach to improve the stability of the training process and facilitate the adaptability of CreINNs to large and deep modern network architectures. Experimental results on an out-of-distribution (OoD) detection benchmark (CIFAR10 vs SVHN) using ResNet50 architecture show that CreINNs outperform variational Bayesian neural networks (BNNs) and deep ensembles (DEs) in terms of EU estimation. CreINNs also exhibit significant computational complexity reduction and smaller model sizes compared to BNNs and DEs. The paper concludes with discussions on future work, including reducing computational complexity and developing a comprehensive evaluation framework for uncertainty-aware models.
Reach us at info@study.space
Understanding CreINNs%3A Credal-Set Interval Neural Networks for Uncertainty Estimation in Classification Tasks