Past exam of the mathematics course of the University of Cambridge / 2002 / iii / Paper 40 / 3 / b

The order prior and probabilities of choosing upward or downward moves are not specified, so include them explicitly. Let ${\rho }_k$ be the model-order prior probability, $b_k$ the birth-selection probability at order $k$, and $d_{k+1}$ the reverse death-selection probability. Denote the correctly normalized coefficient prior by $p_k(a)$ and the probability density function of $z\sim N(0,{\sigma }_a^2)$ by $q(z)$. Common variance-prior factors cancel since $v={\sigma }^2$ is unchanged.

Put $m=n^{-1}{\sum }_i{x}_i^{k+1}$, $r=y-X_{k}a$, and $w_i=x_i^{k+1}-m$. The centered polynomial birth move in reversible-jump sampling gives $X_{k+1}{a}^{\prime }=X_{k}a+zw$, so the residual-sum-of-squares change isThe map $(a_0,a_1,\dots ,a_k,z)\mapsto (a_0-mz,a_1,\dots ,a_k,z)$ is triangular with determinant one. Thus the reversible-jump Markov chain Monte Carlo ratio isFor an entirely explicit Gaussian expression, define $Q_k=(a-{\mu }_k{)}^T{\Sigma }_k^{-1}(a-{\mu }_k)$ and $Q_{k+1}=(a^{\prime }-{\mu }_{k+1}{)}^T{\Sigma }_{k+1}^{-1}(a^{\prime }-{\mu }_{k+1})$. Dividing the two normalized Gaussian priors and the proposal probability density function givesEqual model-order priors and equal birth/death selection probabilities remove the first ratio, if that is the intended convention. Dropping the order-dependent normal-prior constants instead would give the wrong acceptance rule.

From order $k+1$, set $z=a_{k+1}^{\prime }$, remove that final coefficient, retain $a_i=a_i^{\prime }$ for $1\le i\le k$, and restore the intercept byKeep $v$ fixed. This deterministically recovers the lower-order state and the auxiliary variable that would reproduce the higher-order state under a birth. The inverse map also has absolute Jacobian determinant one. In particular, simply deleting the highest coefficient without restoring the intercept is not the reverse of the supplied move.

For the recovered $a,z$ of the preceding part, compute the forward ratio $R$ from part (i). The reverse probability density function ratio and reverse Jacobian determinant invert every factor, soEquivalently, with $L_k(a;v)$ denoting the Gaussian regression likelihood,The factor $q(z)$ is present even though no random variable is generated during the death proposal: it is the probability density function of the auxiliary variable required by the reverse birth. These reciprocal rules give detailed balance on the paired dimension-matching moves.