Use a <singular value decomposition> $X=UDV^T$. Then
$$
(X^TX+\lambda I)^{-1}X^T
=V(D^TD+\lambda I)^{-1}D^TU^T.
$$
Each nonzero singular value $d$ contributes $d/(d^2+\lambda)\to d^{-1}$, while each zero one contributes zero. Thus
$$
\lim_{\lambda\downarrow0}\widehat\beta_\lambda
=VD^+U^TY=(X^TX)^+X^TY,
$$
using the <Moore-Penrose inverse>.

Solution

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 205 / 5 / b / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2025/iii/paper-205/5/b/solution