= Solution
Let $z=\widehat\beta_{\mathrm{OLS}}$ and $G=X^TX/n=\begin{pmatrix}1&\rho\\\rho&1\end{pmatrix}$. While both lasso coordinates are positive, the <Karush-Kuhn-Tucker conditions> give
$$
G(\widehat\beta-z)+\lambda\binom11=0,
\qquad
\widehat\beta=z-\frac{\lambda}{1+\rho}\binom11.
$$
Hence
$$
\lambda^\dagger=(1+\rho)\min(z_1,z_2).
$$
Assume without loss of generality that $z_1\leq z_2$. After the first coordinate vanishes, the second remains active until $lambda=z_2+\rho z_1$. The zero vector satisfies the KKT conditions exactly when $lambda\geq\lVert Gz\rVert_\infty$; for $z_1\leq z_2$ and $-1<\rho<1$, this norm is $z_2+\rho z_1$. Therefore
$$
\lambda^\ddagger=\max(z_1,z_2)+\rho\min(z_1,z_2),
$$
and
$$
\lambda^\ddagger-\lambda^\dagger
=\max(z_1,z_2)-\min(z_1,z_2)=|z_1-z_2|,
$$
which is independent of $\rho$.
Back to article page
