Past exam of the mathematics course of the University of Cambridge / 2002 / iii / Paper 40 / 3 / a / ii / Solution

Write $v={\sigma }^2$, $d=k+1$, $X=X_k$, $a=a_k$, $\mu ={\mu }_k$ and $\Sigma ={\Sigma }_k$, with $\Sigma$ positive-definite. For fixed $v$, the terms involving $a$ in the posterior exponent areComplete the square. With $V_k=(X_k^T{X}_k/v+{\Sigma }_k^{-1}{)}^{-1}$ and $m_k=V_k(X_k^{T}y/v+{\Sigma }_k^{-1}{\mu }_k)$, this givesFor fixed $a_k$, collect the powers and exponential terms involving $v$:Hence, in the paper's shape-scale inverse-gamma distribution convention,There is no extra $(k+1)/2$ in this shape, because the normal coefficient prior is independent of $v$. These are the independent normal and inverse-gamma regression priors conditionals.

The Gaussian normalization printed in the PDF is incorrect. The proper $d$-dimensional prior probability density function is $(2\pi {)}^{-d/2}|{\Sigma }_k{|}^{-1/2}\exp [-(a_k-{\mu }_k{)}^T{\Sigma }_k^{-1}(a_k-{\mu }_k)/2]$. For fixed $k$ this correction is constant in $a_k,v$ and leaves the two conditionals unchanged, but it must be retained when comparing different model orders in part (b).