May 17, 2024 | Matthias Kellner and Michele Ceriotti
Uncertainty quantification (UQ) is essential for data-driven models to assess prediction reliability and improve trustworthiness. This work introduces a practical approach called direct propagation of shallow ensembles (DPOSE) for UQ, which balances accuracy and ease of implementation. DPOSE combines an ensemble architecture with weight sharing, NLL-based training, and ensemble propagation for error estimation. It is particularly effective for atomistic modeling, where uncertainty propagation is critical for reliable predictions.
The paper evaluates various UQ frameworks against four key requirements: accurate uncertainty estimates, minimal computational overhead, ease of implementation, and robust error propagation. DPOSE meets these criteria by using a shallow ensemble with shared weights, reducing computational cost and simplifying implementation. It also enables accurate uncertainty propagation for derived quantities, which is crucial for applications like thermodynamic averages in atomistic simulations.
The study demonstrates DPOSE's effectiveness on diverse datasets, including liquid water, barium titanate, lithium thiophosphate, and the QM9 molecular dataset. It shows that DPOSE provides reliable uncertainty estimates, even for out-of-distribution data, and outperforms other methods in terms of calibration and error propagation. The approach is particularly useful for atomistic modeling, where accurate uncertainty quantification is essential for reliable predictions of properties like energy and forces.
The paper also discusses the importance of post-hoc calibration and the challenges of uncertainty propagation in complex models. It highlights the need for robust calibration strategies and the benefits of using ensemble methods for error estimation. Overall, DPOSE offers a practical and effective solution for UQ in data-driven models, particularly in scientific applications requiring accurate and reliable predictions.Uncertainty quantification (UQ) is essential for data-driven models to assess prediction reliability and improve trustworthiness. This work introduces a practical approach called direct propagation of shallow ensembles (DPOSE) for UQ, which balances accuracy and ease of implementation. DPOSE combines an ensemble architecture with weight sharing, NLL-based training, and ensemble propagation for error estimation. It is particularly effective for atomistic modeling, where uncertainty propagation is critical for reliable predictions.
The paper evaluates various UQ frameworks against four key requirements: accurate uncertainty estimates, minimal computational overhead, ease of implementation, and robust error propagation. DPOSE meets these criteria by using a shallow ensemble with shared weights, reducing computational cost and simplifying implementation. It also enables accurate uncertainty propagation for derived quantities, which is crucial for applications like thermodynamic averages in atomistic simulations.
The study demonstrates DPOSE's effectiveness on diverse datasets, including liquid water, barium titanate, lithium thiophosphate, and the QM9 molecular dataset. It shows that DPOSE provides reliable uncertainty estimates, even for out-of-distribution data, and outperforms other methods in terms of calibration and error propagation. The approach is particularly useful for atomistic modeling, where accurate uncertainty quantification is essential for reliable predictions of properties like energy and forces.
The paper also discusses the importance of post-hoc calibration and the challenges of uncertainty propagation in complex models. It highlights the need for robust calibration strategies and the benefits of using ensemble methods for error estimation. Overall, DPOSE offers a practical and effective solution for UQ in data-driven models, particularly in scientific applications requiring accurate and reliable predictions.