Past exam of the mathematics course of the University of Cambridge / 2020 / ii / Paper 3 / 28J (Principles of Statistics)

Let $K(t,\cdot )$ be the Markov kernel of the Metropolis–Hastings algorithm. Away from the possibility of remaining at $t$, its transition density isMultiplication by the target density givesThis is detailed balance for every pair of distinct states.

Writefor the rejection probability, soFor every measurable $B\subseteq \Theta$, detailed balance and the Tonelli theorem giveHence $\mu$ is an invariant distribution of the chain.

Let $L(t)=f(x,t)$ be the likelihood function. Since the prior distribution is $N(0,I_p)$, the posterior density has the formwhere $Z$ is the normalizing constant.

Choose $0<\delta \le 1/2$ and use the Gaussian autoregressive proposal reversible with respect to a standard normal distributionThuswhose covariance matrix is the required $2\delta I_p$. Reversibility with respect to the standard normal distribution sayssoThe Metropolis–Hastings acceptance probability therefore reduces toThis is the Preconditioned Crank–Nicolson algorithm with proposal scale $\beta =\sqrt{2\delta }$.