= Solution
Put $z=\widehat\beta_{\mathrm{OLS}}=X^TY/n$. Under $X^TX=nI_p$, the objective separates by coordinates. Completing the square and applying the <soft-thresholding operator> $S(u,t)=\operatorname{sign}(u)(|u|-t)_+$ gives
$$
(\widehat\beta_{\alpha,\lambda})_j
=\frac{S(z_j,\lambda\alpha)}{1+\lambda(1-\alpha)}.
$$
For a fixed $z_j$, its magnitude lies between the ridge endpoint $|z_j|/(1+\lambda)$ and the lasso endpoint $(|z_j|-\lambda)_+$; this follows directly on the two intervals $\lambda\alpha\geq|z_j|$ and $\lambda\alpha<|z_j|$ by cross-multiplication. Thus the stated endpoint inequality holds.
For $alpha>0$ and $z_j\ne0$, the coordinate first vanishes when the soft threshold reaches $|z_j|$, so
$$
(\lambda_α^*)_j=\frac{|z_j|}{\alpha}.
$$
This is strictly decreasing in $\alpha$, and it diverges to infinity as $\alpha\downarrow0$. This agrees with the fact that pure ridge shrinkage does not set a nonzero coordinate exactly to zero at any finite penalty.
Back to article page
