Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 47 / 5 / b

Let $v={\sigma }^2>0$, $d_k=k+1$, and $S_k(a)=\Vert y-X_{k}a{\Vert }^2$. Under the standard interpretation that the two stated priors are independent, multiply the normal likelihood by the normal coefficient prior and the inverse-gamma distribution density. The independent normal and inverse-gamma regression priors giveFor a fixed model the determinant and Gaussian normalizing constants are independent of $a_k,v$ and may be omitted here; ${\Sigma }_k$ is assumed positive-definite. The coefficient prior is not scaled by $v$, so replacing its precision by ${\Sigma }_k^{-1}/v$ would describe a different prior.

For an additional check, completing the coefficient square and reading the variance kernel give the full conditional distributionsandMarginal priors alone would not determine their joint prior without the independence assumption. The front-page inverse-gamma mean has a typographical error: this density has mean $\beta /(\alpha -1)$ when $\alpha >1$, not a denominator $\alpha +1$. The density itself, used in the posterior calculation, fixes the correct parameterization.

The cross-model target must include a model-order prior ${\rho }_k$. Write its joint, unnormalized density asHere the prior densities are normalized within each model. In particular their $k$-dependent determinants and powers of $2\pi$ cannot be dropped merely because the fixed-model posterior in part (i) was given up to proportionality.

For explicit birth and death moves for Bayesian variable selection, choose a birth with probability $b_k$, generate $u$ from a density $g_k(u\mid a,v)$, append $u$ as the new covariate coefficient, and leave $v$ unchanged. The map from $(a,v,u)$ to $(a^{\prime },v^{\prime })=((a,u),v)$ is invertible under the reverse deletion and has absolute Jacobian one. It matches dimensions: the smaller-model coefficients and variance have dimension $k+2$, and the one auxiliary draw supplies the larger-model dimension $k+3$. If the reverse death is selected with probability $d_{k+1}$, accept withFor a death, delete the last coefficient $u$, retain the others and the variance, and useTo verify detailed balance, multiply the birth acceptance by $t_k{b}_k{g}_k$ and the reverse death acceptance by $t_{k+1}{d}_{k+1}$. Both equal the minimum of these two quantities in the matched coordinates. This proves reversibility without merely naming a dimension-changing algorithm. A more general bijection introduces its absolute Jacobian, and random choices among several covariates introduce the corresponding forward and reverse selection probabilities. Add within-model updates so that coefficients and variance can explore each model. These are reversible-jump Markov chain Monte Carlo updates; the unspecified model prior and proposal density must be supplied to implement them numerically.