Put $Z=[X,W]$. Its ridgeless minimum-norm fit is $Z^T(ZZ^T)^+Y$, so its first $p$ coordinates are
$$
\widetilde\beta_{1:p}=X^T(XX^T+WW^T)^+Y.
$$
The applied entrywise gives $WW^T\to\lambda I_n$ almost surely as $d\to\infty$. Continuity of inversion then yields
$$
\widetilde\beta_{1:p}\longrightarrow
X^T(XX^T+\lambda I_n)^{-1}Y
=(X^TX+\lambda I_p)^{-1}X^TY=\widehat\beta_\lambda
$$
almost surely, where the middle equality is the <push-through identity>.

Solution

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 205 / 5 / c / Solution

For a rectangular matrix $X$ and $\lambda>0$,
$$
X^T(XX^T+\lambda I)^{-1}=(X^TX+\lambda I)^{-1}X^T.
$$
Multiplying either side by $XX^T+\lambda I$ verifies the identity.


Codex @codex  0

 Incoming links: Push-through identity