Predictive Uncertainty Estimation via Prior Networks

Predictive Uncertainty Estimation via Prior Networks

29 Nov 2018 | Andrey Malinin, Mark Gales
This paper introduces Prior Networks (PNs), a new framework for modeling predictive uncertainty in AI systems. Unlike previous methods that implicitly model uncertainty through model or data uncertainty, PNs explicitly model distributional uncertainty by parameterizing a prior distribution over predictive distributions. This allows for distinguishing between data uncertainty, model uncertainty, and distributional uncertainty, which is crucial for making informed decisions in AI applications. The work focuses on classification tasks and evaluates PNs on the tasks of identifying out-of-distribution (OOD) samples and detecting misclassification on the MNIST and CIFAR-10 datasets, where they outperform previous methods. Experiments on synthetic and MNIST data show that PNs are able to distinguish between data and distributional uncertainty, which is not possible with previous non-Bayesian methods. PNs are also evaluated on the tasks of identifying OOD samples and detecting misclassification, where they outperform previous methods on the MNIST and CIFAR-10 datasets. The paper discusses various uncertainty measures and their application in different contexts, and shows that differential entropy of the DPN is best for OOD detection, especially when classes are less distinct. The work concludes that PNs provide a more accurate and interpretable way to model predictive uncertainty compared to previous methods.This paper introduces Prior Networks (PNs), a new framework for modeling predictive uncertainty in AI systems. Unlike previous methods that implicitly model uncertainty through model or data uncertainty, PNs explicitly model distributional uncertainty by parameterizing a prior distribution over predictive distributions. This allows for distinguishing between data uncertainty, model uncertainty, and distributional uncertainty, which is crucial for making informed decisions in AI applications. The work focuses on classification tasks and evaluates PNs on the tasks of identifying out-of-distribution (OOD) samples and detecting misclassification on the MNIST and CIFAR-10 datasets, where they outperform previous methods. Experiments on synthetic and MNIST data show that PNs are able to distinguish between data and distributional uncertainty, which is not possible with previous non-Bayesian methods. PNs are also evaluated on the tasks of identifying OOD samples and detecting misclassification, where they outperform previous methods on the MNIST and CIFAR-10 datasets. The paper discusses various uncertainty measures and their application in different contexts, and shows that differential entropy of the DPN is best for OOD detection, especially when classes are less distinct. The work concludes that PNs provide a more accurate and interpretable way to model predictive uncertainty compared to previous methods.
Reach us at info@study.space
Understanding Predictive Uncertainty Estimation via Prior Networks