Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 47 / 5 / b / i

Let

v = σ^{2} > 0

d_{k} = k + 1

, and

S_{k} (a) = ∥ y - X_{k} a ∥^{2}

. Under the standard interpretation that the two stated priors are independent, multiply the normal likelihood by the normal coefficient prior and the inverse-gamma distribution density. The independent normal and inverse-gamma regression priors give

π (a_{k}, v ∣ x, y) \propto v^{- (α + 1 + n /2)} exp [- \frac{β + S _{k} ( a _{k} ) /2}{v} - \frac{1}{2} (a_{k} - μ_{k})^{T} Σ_{k}^{- 1} (a_{k} - μ_{k})], v > 0.

(1)

For a fixed model the determinant and Gaussian normalizing constants are independent of

a_{k}, v

and may be omitted here;

Σ_{k}

is assumed positive-definite. The coefficient prior is not scaled by

v

, so replacing its precision by

Σ_{k}^{- 1} / v

would describe a different prior.

For an additional check, completing the coefficient square and reading the variance kernel give the full conditional distributions

a_{k} ∣ v, x, y \sim N (m_{k}, V_{k}), V_{k} = (X_{k}^{T} X_{k} / v + Σ_{k}^{- 1})^{- 1}, m_{k} = V_{k} (X_{k}^{T} y / v + Σ_{k}^{- 1} μ_{k}),

(2)

and

v ∣ a_{k}, x, y \sim InvGamma (α + n /2, β + S_{k} (a_{k}) /2) .

(3)

Marginal priors alone would not determine their joint prior without the independence assumption. The front-page inverse-gamma mean has a typographical error: this density has mean