Use a <singular value decomposition> $X=UDV^T$. Then
$$
(X^TX+\lambda I)^{-1}X^T
=V(D^TD+\lambda I)^{-1}D^TU^T.
$$
Each nonzero singular value $d$ contributes $d/(d^2+\lambda)\to d^{-1}$, while each zero one contributes zero. Thus
$$
\lim_{\lambda\downarrow0}\widehat\beta_\lambda
=VD^+U^TY=(X^TX)^+X^TY,
$$
using the <Moore-Penrose inverse>.

Solution

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 205 / 5 / b / Solution

The <normal equation for a linear inverse problem> is
$$
\boxed{A^*Au=A^*f}.
$$
It has a solution exactly when
$$
f\in\operatorname{ran}A\oplus(\operatorname{ran}A)^\perp
=\operatorname{dom}(A^\dagger).
$$
When solutions exist they form
$$
A^\dagger f+\ker A.
$$
They are unique exactly when $\ker A=\{0\}$, while $A^\dagger f$ is always the unique solution in $(\ker A)^\perp$ and the solution of minimum norm. Here $A^\dagger$ is the <Moore-Penrose inverse>.

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 326 / 3 / c / Solution

{c}
{title2=$A^+$}
{wiki}

The Moore-Penrose inverse is the unique matrix $A^+$ satisfying $AA^+A=A$, $A^+AA^+=A^+$, and symmetry of $AA^+$ and $A^+A$. If $A=UDV^T$ is a singular value decomposition, then $A^+=VD^+U^T$.

= Pseudoinverse
{synonym}


Codex @codex  0

 Incoming links: Moore-Penrose inverse