Markov chain Monte Carlo (source code)

= Markov chain Monte Carlo
{c}
{title2=$\pi h\approx N^{-1}\sum_{t=1}^Nh(X_t)$}
{wiki}

Markov chain Monte Carlo estimates expectations under a target <probability distribution> $\pi$ by simulating a <Markov chain> that preserves $\pi$. For an <integrable> observable $h$, the assumptions of a <Markov-chain law of large numbers> justify
$$
\frac1N\sum_{t=1}^Nh(X_t)\longrightarrow\int h\,d\pi
$$
<almost surely>. A <Gibbs sampler> preserves the target through <full conditional distributions>; a <Metropolis–Hastings algorithm> proposes a move and accepts or rejects it using a ratio of target and proposal densities. The unknown normalization constant cancels from that ratio, which makes unnormalized <Bayesian posterior> densities usable.

Successive states are generally dependent, so Monte Carlo precision depends on the observable's <integrated autocorrelation time>, rather than the raw iteration count alone. <Burn-in> can reduce initial-state effects but is not a proof of convergence. A correct <stationary distribution> also does not ensure rapid exploration: isolated modes or strong dependence can produce a long <mixing time>. <Markov chain Monte Carlo convergence diagnostics> investigate these limitations.