Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 216 / 2

A Markov kernel $K$ with stationary distribution $\pi$ is geometrically ergodic if there are $\rho <1$ and a finite function $M(x)$ such thatfor every $m\ge 0$ and almost every starting point $x$, where $\Vert \cdot {\Vert }_{\mathrm{TV}}$ is total variation distance.

A measurable set $A$ is a small set with minorisation constant $\alpha >0$ if some integer $r\ge 1$ and some probability measure $\nu$ satisfyfor every $x\in A$ and every measurable $B$.

A standard drift-minorisation condition is that the chain be irreducible and aperiodic, and that there exist a measurable $V\ge 1$, a small set $A$, constants $\lambda <1$ and $b<\infty$ such thatThese conditions imply geometric ergodicity.

Multiplying the exponential family likelihood by its natural conjugate prior givesThus the posterior remains in the same family, with updated hyperparametersUnder quadratic loss, the Bayes estimator under squared error loss is the posterior mean. Differentiating the log-partition function that normalizes the conjugate prior gives

Let $w(\theta )=\pi (\theta )/\mu (\theta )$ be the importance weight. The two normalized densities giveSince $|F-Z|<C$, there is a finite constant $M$ such that $w(\theta )\le M$ everywhere. The Independence Metropolis–Hastings algorithm has an accepted transition density satisfyingConsequently the whole state space is a small set, with the one-step minorization condition $K(x,\cdot )\ge M^{-1}\pi (\cdot )$. Iterating this Doeblin condition gives uniform geometric convergence in total variation distance, so the chain is geometrically ergodic.