March 2001, Vol. 96, No. 453 | Siddhartha Chib and Ivan Jeliazkov
This article presents a method for estimating the marginal likelihood in Bayesian model comparison using MCMC output from the Metropolis-Hastings algorithm. The approach extends Chib's (1995) method by addressing issues related to intractable full conditional densities. The method leverages MCMC samples for both parameter estimation and marginal likelihood computation, reducing the need for separate tuning. The method is demonstrated through examples including the logit model for binary data, hierarchical random effects model for clustered Gaussian data, Poisson regression for clustered count data, and multivariate probit for correlated binary data. These examples show the method's practicality and wide applicability.
The marginal likelihood is calculated using the basic marginal likelihood identity, which relates the marginal likelihood to the posterior density at a specific point. The method involves estimating the posterior ordinate at this point using MCMC samples. For models with intractable full conditional densities, the method uses a reduced MCMC run to estimate the posterior ordinate. The approach is flexible and can handle both low- and high-dimensional problems by dividing parameters into blocks. The method is also applicable to models with latent variables and multiple blocks.
The article discusses the numerical standard error of the marginal likelihood estimate, showing how it can be derived using the Delta method. The method is validated through examples, demonstrating its effectiveness in estimating the marginal likelihood with high accuracy and low numerical standard error. The approach is efficient for both parameter sampling and marginal likelihood estimation, making it a valuable tool for Bayesian model comparison.This article presents a method for estimating the marginal likelihood in Bayesian model comparison using MCMC output from the Metropolis-Hastings algorithm. The approach extends Chib's (1995) method by addressing issues related to intractable full conditional densities. The method leverages MCMC samples for both parameter estimation and marginal likelihood computation, reducing the need for separate tuning. The method is demonstrated through examples including the logit model for binary data, hierarchical random effects model for clustered Gaussian data, Poisson regression for clustered count data, and multivariate probit for correlated binary data. These examples show the method's practicality and wide applicability.
The marginal likelihood is calculated using the basic marginal likelihood identity, which relates the marginal likelihood to the posterior density at a specific point. The method involves estimating the posterior ordinate at this point using MCMC samples. For models with intractable full conditional densities, the method uses a reduced MCMC run to estimate the posterior ordinate. The approach is flexible and can handle both low- and high-dimensional problems by dividing parameters into blocks. The method is also applicable to models with latent variables and multiple blocks.
The article discusses the numerical standard error of the marginal likelihood estimate, showing how it can be derived using the Delta method. The method is validated through examples, demonstrating its effectiveness in estimating the marginal likelihood with high accuracy and low numerical standard error. The approach is efficient for both parameter sampling and marginal likelihood estimation, making it a valuable tool for Bayesian model comparison.