The paper introduces a deterministic variational formulation for training Bayesian last layer neural networks, known as Variational Bayesian Last Layers (VBLL). This formulation allows for a sampling-free, single-pass model and loss, improving uncertainty estimation. VBLLs can be trained and evaluated with quadratic complexity in the width of the last layer, making them computationally efficient and easy to integrate into standard architectures. The authors experimentally demonstrate that VBLLs improve predictive accuracy, calibration, and out-of-distribution detection across regression and classification tasks. They also explore combining VBLL layers with variational Bayesian feature learning, leading to a lower variance collapsed variational inference method for Bayesian neural networks. The paper includes detailed derivations of variational objectives for regression, discriminative classification, and generative classification models, and provides experimental results showing the effectiveness of VBLLs in various settings, including image classification, sentiment analysis, and active learning.The paper introduces a deterministic variational formulation for training Bayesian last layer neural networks, known as Variational Bayesian Last Layers (VBLL). This formulation allows for a sampling-free, single-pass model and loss, improving uncertainty estimation. VBLLs can be trained and evaluated with quadratic complexity in the width of the last layer, making them computationally efficient and easy to integrate into standard architectures. The authors experimentally demonstrate that VBLLs improve predictive accuracy, calibration, and out-of-distribution detection across regression and classification tasks. They also explore combining VBLL layers with variational Bayesian feature learning, leading to a lower variance collapsed variational inference method for Bayesian neural networks. The paper includes detailed derivations of variational objectives for regression, discriminative classification, and generative classification models, and provides experimental results showing the effectiveness of VBLLs in various settings, including image classification, sentiment analysis, and active learning.