= Solution
For a target <probability density function> $\pi$ and a <proposal distribution> with density $q(x'\mid x)$, the <Metropolis–Hastings algorithm> starts in the target support and repeats: propose $x'\sim q(\cdot\mid x)$, draw an <independent> uniform $U$, and move to $x'$ if
$$
U\le\alpha(x,x'),\qquad\boxed{\alpha(x,x')=\min\left(1,\frac{\pi(x')q(x\mid x')}{\pi(x)q(x'\mid x)}\right).}
$$
Otherwise retain $x$. A zero reverse-proposal density makes the acceptance zero. Unknown multiplicative constants of $\pi$ cancel. For distinct states,
$$
\pi(x)q(x'\mid x)\alpha(x,x')=\min\{\pi(x)q(x'\mid x),\pi(x')q(x\mid x')\},
$$
which is symmetric in the states. Together with the probability of holding at the current state, this proves <detailed balance> and invariance of $\pi$. Successive values form a dependent <Markov chain>; convergence from arbitrary initial states additionally needs the appropriate irreducibility and <aperiodicity>, not just this invariant-density identity.
The <Gibbs sampler> updates a coordinate using its <full conditional distribution> while keeping the others fixed. For coordinate $i$, write $r=x_{-i}$ and $\pi(x)=\pi_{-i}(r)\pi_i(x_i\mid r)$. Its proposal replaces $x_i$ by $x_i'\sim\pi_i(\cdot\mid r)$. The MH ratio, interpreted on this one-coordinate proposal space, is
$$
\frac{\pi_{-i}(r)\pi_i(x_i'\mid r)\pi_i(x_i\mid r)}{\pi_{-i}(r)\pi_i(x_i\mid r)\pi_i(x_i'\mid r)}=1.
$$
Thus \b[every Gibbs coordinate update is an MH update with acceptance one]. A random choice of coordinate gives a mixture of these invariant kernels; a deterministic sweep gives their composition. Both preserve $\pi$. The latter composition need not be reversible, so the special-case statement is about the coordinate kernels, not a claim that every entire systematic sweep is one reversible MH kernel.
Back to article page
