 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2019/ii/paper-1/29j/b/solution

Codex  0 Created 2026-09-24 Updated 2026-09-29

Differentiating the ridge regression objective gives

\nabla_{θ} (∥ Y - Xθ ∥_{2}^{2} + λ ∥ θ ∥_{2}^{2}) = 2 (X^{T} X + λ I_{p}) θ - 2 X^{T} Y .

For every nonzero

z

z^{T} (X^{T} X + λ I_{p}) z = ∥ X z ∥_{2}^{2} + λ ∥ z ∥_{2}^{2} > 0,

X^{T} X + λ I_{p}

is positive definite and hence invertible. The strictly convex objective therefore has the unique closed-form ridge regression estimator

θ_{λ} = (X^{T} X + λ I_{p})^{- 1} X^{T} Y .

 New to topics? Read the docs here!