6 Aug 2015 | Daniel Simpson*, Håvard Rue, Thiago G. Martins, Andrea Riebler, and Sigrunn H. Sørbye
This paper introduces a principled and practical approach to constructing prior distributions for hierarchical models, known as Penalised Complexity (PC) priors. These priors are designed to penalize model complexity by deviating from a simpler base model, and are defined using a user-specified scaling parameter. The approach is invariant to reparameterisations, has a natural connection to Jeffreys' priors, supports Occam's razor, and exhibits robustness to the choice of flexibility parameters. PC priors are constructed based on four principles: Occam's razor, measure of complexity, constant rate penalisation, and user-defined scaling. These priors are particularly useful for hierarchical models with multiple components, as they allow for flexible model extensions while controlling overfitting. The paper demonstrates the appropriateness of PC priors through examples and theoretical results, showing that they can be applied in various situations. The approach is not restricted to any specific computational method and is relevant to any Bayesian analysis. The paper also discusses the challenges of constructing non-subjective priors for hierarchical models, highlighting the limitations of objective and weakly informative priors. It emphasizes the importance of considering the model structure when specifying priors and the need for computationally feasible and theoretically sound methods. The paper concludes that PC priors provide a useful and flexible framework for constructing priors in hierarchical models, with desirable properties such as invariance, connection to Jeffreys' priors, and robustness. The paper also discusses the importance of considering the base model in prior specification and the need for priors that control flexibility and avoid overfitting. The paper highlights the challenges of applying priors in high-dimensional models and the importance of tail behaviour in ensuring robustness. The paper concludes that PC priors offer a principled and practical approach to constructing priors for hierarchical models, with desirable properties that make them suitable for a wide range of applications.This paper introduces a principled and practical approach to constructing prior distributions for hierarchical models, known as Penalised Complexity (PC) priors. These priors are designed to penalize model complexity by deviating from a simpler base model, and are defined using a user-specified scaling parameter. The approach is invariant to reparameterisations, has a natural connection to Jeffreys' priors, supports Occam's razor, and exhibits robustness to the choice of flexibility parameters. PC priors are constructed based on four principles: Occam's razor, measure of complexity, constant rate penalisation, and user-defined scaling. These priors are particularly useful for hierarchical models with multiple components, as they allow for flexible model extensions while controlling overfitting. The paper demonstrates the appropriateness of PC priors through examples and theoretical results, showing that they can be applied in various situations. The approach is not restricted to any specific computational method and is relevant to any Bayesian analysis. The paper also discusses the challenges of constructing non-subjective priors for hierarchical models, highlighting the limitations of objective and weakly informative priors. It emphasizes the importance of considering the model structure when specifying priors and the need for computationally feasible and theoretically sound methods. The paper concludes that PC priors provide a useful and flexible framework for constructing priors in hierarchical models, with desirable properties such as invariance, connection to Jeffreys' priors, and robustness. The paper also discusses the importance of considering the base model in prior specification and the need for priors that control flexibility and avoid overfitting. The paper highlights the challenges of applying priors in high-dimensional models and the importance of tail behaviour in ensuring robustness. The paper concludes that PC priors offer a principled and practical approach to constructing priors for hierarchical models, with desirable properties that make them suitable for a wide range of applications.