MCMC using Hamiltonian dynamics

MCMC using Hamiltonian dynamics

2011 | Radford M. Neal, University of Toronto
Hamiltonian dynamics can be used to generate distant proposals for the Metropolis algorithm, thereby avoiding the slow exploration of the state space that results from the diffusive behavior of simple random-walk proposals. Hamiltonian dynamics, which originated in physics, can be applied to most problems with continuous state spaces by introducing fictitious "momentum" variables. A key advantage of Hamiltonian dynamics is that it preserves volume, allowing complex mappings to be defined without the need to account for a hard-to-compute Jacobian factor. This review discusses the theoretical and practical aspects of Hamiltonian Monte Carlo (HMC) and presents several variations, including using windows of states for acceptance or rejection, computing trajectories using fast approximations, tempering during the trajectory to handle isolated modes, and shortcut methods to prevent useless trajectories from consuming computation time. The review also covers the reversibility, volume preservation, and symplectiveness properties of Hamiltonian dynamics, and provides a detailed explanation of how these properties are maintained even when the dynamics is approximated by discretizing time. The leapfrog method, a modification of Euler's method, is introduced as a more accurate and stable way to approximate Hamiltonian dynamics, preserving volume exactly. The Hamiltonian Monte Carlo algorithm is described, including the sampling of momentum variables and the Metropolis update using Hamiltonian dynamics. The review concludes with illustrations demonstrating the benefits of HMC over simple random-walk Metropolis methods, particularly in high-dimensional and correlated distributions.Hamiltonian dynamics can be used to generate distant proposals for the Metropolis algorithm, thereby avoiding the slow exploration of the state space that results from the diffusive behavior of simple random-walk proposals. Hamiltonian dynamics, which originated in physics, can be applied to most problems with continuous state spaces by introducing fictitious "momentum" variables. A key advantage of Hamiltonian dynamics is that it preserves volume, allowing complex mappings to be defined without the need to account for a hard-to-compute Jacobian factor. This review discusses the theoretical and practical aspects of Hamiltonian Monte Carlo (HMC) and presents several variations, including using windows of states for acceptance or rejection, computing trajectories using fast approximations, tempering during the trajectory to handle isolated modes, and shortcut methods to prevent useless trajectories from consuming computation time. The review also covers the reversibility, volume preservation, and symplectiveness properties of Hamiltonian dynamics, and provides a detailed explanation of how these properties are maintained even when the dynamics is approximated by discretizing time. The leapfrog method, a modification of Euler's method, is introduced as a more accurate and stable way to approximate Hamiltonian dynamics, preserving volume exactly. The Hamiltonian Monte Carlo algorithm is described, including the sampling of momentum variables and the Metropolis update using Hamiltonian dynamics. The review concludes with illustrations demonstrating the benefits of HMC over simple random-walk Metropolis methods, particularly in high-dimensional and correlated distributions.
Reach us at info@study.space
[slides] MCMC using Hamiltonian dynamics | StudySpace