Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 216 / 4

Hamiltonian Monte Carlo augments the position $x$ by an independent momentum $p\sim N(0,M)$ and uses the Hamiltonian functionFrom the current $x$, draw a fresh $p$, apply a fixed number of leapfrog steps to approximate Hamiltonian flow, and obtain $(x^{\prime },p^{\prime })$. The Metropolis–Hastings acceptance probability isotherwise retain $x$. Momentum negation may be included to make the proposal explicitly reversible. The leapfrog map is volume preserving and reversible, while the acceptance step corrects its discretization error, leaving $\pi$ invariant after the momentum is discarded.

For a sufficiently regular Itô diffusion with $D=\sigma {\sigma }^T/2$, a density $\pi$ is stationary if and only if it solves the stationary Fokker-Planck equationwith integrable Fokker-Planck probability current and boundary conditions that make its outward flux vanish. Here $(\nabla \cdot (D\pi ){)}_i={\sum }_j{\partial }_j(D_{ij}\pi )$.

For the displayed parametrization, assumewhere $Z<\infty$, that $D(x)$ is symmetric positive semidefinite, and that $Q(x)$ is antisymmetric. Substituting $\nabla \pi =-\pi \nabla H$ and the stated $\Gamma$ into the probability current cancels all $D$ terms. The remaining divergence isbecause the second derivatives are symmetric in $i,j$ whereas $Q_{ij}=-Q_{ji}$. Thus these conditions, together with the boundary and regularity assumptions, imply stationarity. More generally, the divergence equation above is the exact necessary and sufficient condition; within this construction, $\pi \propto e^{-H}$ and antisymmetric $Q$ are the standard way to satisfy it.

Invariant distribution of an Itô diffusion, specialized to Underdamped Langevin dynamics, has densityThus $X$ has density proportional to $e^{-U(x)}$, and conditionally and marginally $P\sim N(0,\eta I_d)$. The Hamiltonian transport between $x$ and $p$ preserves this density, while the Ornstein-Uhlenbeck process in momentum has exactly that Gaussian invariant law.

The Euler-Maruyama method with step size $\delta$ and independent ${\xi }_k\sim N(0,I_d)$ is