Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 218 / 2 / d

Let $z={\hat{\beta }}_{\mathrm{OLS}}$ and $G=X^{T}X/n=\left(\begin{array}{cc}1& \rho \\ \rho & 1\end{array}\right)$. While both lasso coordinates are positive, the Karush-Kuhn-Tucker conditions giveHenceAssume without loss of generality that $z_1\le z_2$. After the first coordinate vanishes, the second remains active until $lambda=z_2+\rho z_1$. The zero vector satisfies the KKT conditions exactly when $lambda\ge \Vert Gz{\Vert }_{\infty }$; for $z_1\le z_2$ and $-1<\rho <1$, this norm is $z_2+\rho z_1$. Thereforeandwhich is independent of $\rho$.