Solution
ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2015/iii/paper-33/4/a/solution
Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 33 4 a Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-06
Use an unpenalized intercept in ridge regression. For centered predictor columns, the ridge regression optimization isDifferentiating with respect to gives . Put . Differentiation with respect to gives the ridge normal equationConsequently the closed-form ridge regression estimator isBecause , the numerator may also be written . For the inverse exists even with multicollinearity or deficient column rank; at a unique ordinary least squares coefficient vector requires full rank. The quadratic penalty stabilizes nearly singular directions but generally does not set individual coefficients exactly to zero.
For uncentered original predictors, use , apply the displayed estimator to , and recover . Any internal scaling used by
lm.ridge must be undone to report coefficients on the original predictor scale. Multiplying the objective by a constant changes the numerical penalty convention unless is rescaled as well. New to topics? Read the docs here!