= Solution
A systematic <Gibbs sampler> updates the coordinates of $\theta$ successively from their <full conditional distributions>. At iteration $t$, draw
$$
\theta_j^{(t+1)}\sim\pi\left(\theta_j\mid
\theta_1^{(t+1)},\ldots,\theta_{j-1}^{(t+1)},
\theta_{j+1}^{(t)},\ldots,\theta_p^{(t)}\right),\qquad j=1,\ldots,p.
$$
For a single coordinate, if the current state has distribution $\pi$, its untouched coordinates have the correct marginal law and the resampled coordinate has precisely the conditional law under $\pi$. Integrating marginal times conditional therefore recovers $\pi$. Each coordinate kernel preserves this <invariant distribution>, so their composition also preserves it. This proves the target of the <Gibbs sampler>; the whole systematic sweep need not be a reversible kernel.
Starting from a suitable state and assuming the required irreducibility and convergence conditions, discard an initial transient and retain subsequent states as a dependent sample from, or approximately from, $\pi$. In finite-state problems irreducibility and aperiodicity suffice; in continuous-state problems the corresponding positive recurrence and ergodicity conditions must hold. Successive draws are generally dependent, so Monte Carlo uncertainty must account for that dependence.
Back to article page
