31 Oct 2018 | Murat Sensoy, Lance Kaplan, Melih Kandemir
This paper proposes a novel approach to modeling prediction uncertainty in deep neural networks using the theory of subjective logic. Unlike Bayesian neural networks that infer uncertainty through weight uncertainties, the proposed method explicitly models uncertainty by treating class probabilities as parameters of a Dirichlet distribution. The neural network's output is used to update the Dirichlet parameters, which represent the evidence leading to the predictions. This approach allows for a more detailed uncertainty model compared to the point estimates provided by standard softmax outputs.
The authors demonstrate that their method outperforms state-of-the-art Bayesian neural networks in two key areas: detecting out-of-distribution queries and robustness against adversarial perturbations. They achieve this by minimizing a loss function that combines the L2-Norm loss with an information-theoretic complexity term, ensuring both accurate predictions and reliable uncertainty estimates.
Experiments on datasets like MNIST and CIFAR10 show that the proposed method provides more accurate uncertainty estimates, leading to better performance in out-of-distribution detection and adversarial robustness. The paper also includes theoretical propositions to support the effectiveness of the proposed loss function and discusses related work in uncertainty modeling.This paper proposes a novel approach to modeling prediction uncertainty in deep neural networks using the theory of subjective logic. Unlike Bayesian neural networks that infer uncertainty through weight uncertainties, the proposed method explicitly models uncertainty by treating class probabilities as parameters of a Dirichlet distribution. The neural network's output is used to update the Dirichlet parameters, which represent the evidence leading to the predictions. This approach allows for a more detailed uncertainty model compared to the point estimates provided by standard softmax outputs.
The authors demonstrate that their method outperforms state-of-the-art Bayesian neural networks in two key areas: detecting out-of-distribution queries and robustness against adversarial perturbations. They achieve this by minimizing a loss function that combines the L2-Norm loss with an information-theoretic complexity term, ensuring both accurate predictions and reliable uncertainty estimates.
Experiments on datasets like MNIST and CIFAR10 show that the proposed method provides more accurate uncertainty estimates, leading to better performance in out-of-distribution detection and adversarial robustness. The paper also includes theoretical propositions to support the effectiveness of the proposed loss function and discusses related work in uncertainty modeling.