Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 218 / 5 / a

For fixed ${\lambda }_2$, letting ${\lambda }_1\downarrow 0$ gives ridge regression, including ordinary least squares when ${\lambda }_2=0$; letting ${\lambda }_1\to \infty$ forces every coefficient to zero. For fixed ${\lambda }_1$, letting ${\lambda }_2\downarrow 0$ gives the Lasso. Letting both penalties vanish gives an ordinary least squares solution, unique when $X$ has full column rank and otherwise potentially nonunique or path-dependent.

When ${\lambda }_2>0$, the term ${\lambda }_2\Vert \beta {\Vert }_2^2$ is strictly convex. Its sum with the convex squared loss and ${\ell }^1$ penalty is strictly convex and coercive, so the elastic net solution exists and is unique.