Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 205 / 6 / Solution

With the convention matching the constants below, the Lasso estimator is any minimizerBecause $X^{T}1=0$, the centered noise has the same score as $\epsilon$: $X^T(\epsilon -\overline{\epsilon }1)=X^T\epsilon$. Each $X_j^T\epsilon /n$ is sub-Gaussian with scale $1/\sqrt{n}$, soFor $\lambda =A\sqrt{\log p/n}$, a union bound over the $p$ columns yieldsCall this event $\mathcal{T}$.

The Karush-Kuhn-Tucker conditions for the Lasso say that there is $\hat{z}\in {\mathbb{R}}^p$ such thatEquivalently, $\hat{z}$ belongs to the subdifferential of the ${\ell }^1$ norm at $\hat{\beta }$.