The <conditional multivariate normal distribution> gives
$$
y\mid z\sim N(z^T\beta,\sigma^2),
\quad
\beta=\Sigma_{-1,-1}^{-1}\Sigma_{-1,1},
\quad
\sigma^2=\Sigma_{11}-\Sigma_{1,-1}\Sigma_{-1,-1}^{-1}\Sigma_{-1,1}>0.
$$
For a jointly Gaussian vector, the residual is independent of the regressor, so $y=z^T\beta+\sigma e$ with $e\sim N(0,1)$ independent of $z$.

The <Square-root Lasso> minimizes
$$
\frac1{\sqrt n}\lVert X-Z\theta\rVert_2+\gamma\lVert\theta\rVert_1,
\qquad \gamma=\sqrt{\frac{2\log p}{n}}.
$$
At a nonzero residual $R$, its KKT condition is
$$
\frac{Z^TR}{\sqrt n\lVert R\rVert_2}=\gamma u,
\qquad\lVert u\rVert_\infty\leq1,
$$
which proves the required inequality.

Writing the response vector as $Y=f+\sigma e$, the reverse triangle inequality gives
$$
\left|\widehat\sigma-\sigma\frac{\lVert e\rVert_2}{\sqrt n}\right|
\leq\frac{\lVert\widehat f-f\rVert_2}{\sqrt n}=o_P(1).
$$
The gives $\lVert e\rVert_2/\sqrt n\to_P1$, hence $\widehat\sigma\to_P\sigma$.

Under $X\perp\!\!\!\perp Y\mid Z$, Gaussianity makes $e$ independent of $(X,Z)$ and hence of $R$. Conditionally on $R$, $R^Te/\lVert R\rVert_2\sim N(0,1)$. The remaining numerator term obeys
$$
\frac{|R^TZ(\beta-\widehat\beta)|}{\lVert R\rVert_2}
\leq\frac{\lVert Z^TR\rVert_\infty}{\lVert R\rVert_2}
\lVert\widehat\beta-\beta\rVert_1
\leq\sqrt{2\log p}\,\lVert\widehat\beta-\beta\rVert_1=o_P(1).
$$
Combining this with $\widehat\sigma\to_P\sigma$ and <Slutsky theorem> proves the standard-normal limit.

Solved by gpt-5.6-sol high.

Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 205 / 5 / Solution

{c}
{wiki=Square-root_lasso}

The square-root Lasso uses the unsquared residual norm plus an $\ell^1$ penalty.


Codex @codex  0

 Incoming links: Square-root Lasso