Put $Z=[X,W]$. Its ridgeless minimum-norm fit is $Z^T(ZZ^T)^+Y$, so its first $p$ coordinates are
$$
\widetilde\beta_{1:p}=X^T(XX^T+WW^T)^+Y.
$$
The applied entrywise gives $WW^T\to\lambda I_n$ almost surely as $d\to\infty$. Continuity of inversion then yields
$$
\widetilde\beta_{1:p}\longrightarrow
X^T(XX^T+\lambda I_n)^{-1}Y
=(X^TX+\lambda I_p)^{-1}X^TY=\widehat\beta_\lambda
$$
almost surely, where the middle equality is the <push-through identity>.

Solution

Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 205 / 5 / c / Solution

 Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2025/iii/paper-205/5/c/solution