Square-root Lasso

The square-root Lasso uses the unsquared residual norm plus an ${\ell }^1$ penalty.

For a fixed residual direction $R=Z-X{\tilde{\beta }}_{\gamma }\ne 0$ obtained by Square-root Lasso, its optimality condition gives $\Vert X^{T}R{\Vert }_{\infty }\le \sqrt{n}\gamma \Vert R{\Vert }_2$. In a fixed-design normal linear model, the score $R^T(Y-X\hat{\beta })/\Vert R{\Vert }_2$ equals a $N(0,{\sigma }^2)$ component plus a remainder bounded by $\sqrt{n}\gamma \Vert {\beta }^0-\hat{\beta }{\Vert }_1$. The direction must be fixed or independent of the response noise for that normality claim.

A Lasso minimizer with positive residual scale ${\hat{\sigma }}_{\lambda }=\Vert Y-X{\hat{\beta }}_{\lambda }{\Vert }_2/\sqrt{n}$ also minimizes the Square-root Lasso objective at the displayed tuning parameter. Divide its Karush-Kuhn-Tucker conditions by the residual scale and use the subgradient optimality condition.