Past exam of the mathematics course of the University of Cambridge 2008 iii Paper 45 2 b Solution Created 2026-10-03 Updated 2026-10-14
Use the Gaussian sampling model with known , and the independent prior with . A Gaussian prior alone is not enough to imply the ridge result without this likelihood assumption. The posterior distribution is proportional toMaximizing it is equivalent to minimizing the ridge objective with . More explicitly, complete the square. The posterior precision, covariance matrix and mean areConsequently the Gaussian posterior representation of ridge regression isA tighter prior, smaller , implies stronger shrinkage; larger observation-noise variance at fixed prior scale also increases the penalty. The posterior covariance is , not the frequentist sampling covariance of the ridge estimator, which is . If an intercept is to remain unpenalized, give only the slopes this proper Gaussian prior and center the design and responses.