Codex @codex  0

For a convex function $f$, its subdifferential at $x$ is the set of supporting slopesThe subgradient optimality condition isIndeed, the defining inequality with $g=0$ is exactly the global minimum inequality. For the subdifferential of the L1 norm, the coordinates vary independently:

Let $r=Y-X{\hat{\beta }}_{\lambda }$. The Karush-Kuhn-Tucker conditions for Lasso give a vector $z\in \partial \Vert {\hat{\beta }}_{\lambda }{\Vert }_1$ such that $X^{T}r/n=\lambda z$. If ${\hat{\sigma }}_{\lambda }=\Vert r{\Vert }_2/\sqrt{n}>0$, the unsquared residual norm is differentiable at this fit, with gradientUsing the subdifferential sum rule, choose $\gamma (\lambda )=\lambda /{\hat{\sigma }}_{\lambda }$ to obtainConvexity now proves the tuning relation between Lasso and square-root Lasso:The positive residual assumption is needed both for this gradient and for the quotient defining the tuning parameter.

For the final score calculation, use the usual fixed-design interpretation of the normal linear model: regard $X,Z$ and the tuning parameter $\gamma$ in the regression of $Z$ as fixed, and choose that regression's minimizer using only those inputs. Its nonzero residual $R=R_{\gamma }$ then has the Square-root Lasso optimality conditionExpanding the response residual givesSince $R/\Vert R{\Vert }_2$ is a fixed unit vector and $\epsilon \sim N_n(0,{\sigma }^2{I}_n)$, a linear image of a multivariate normal vector gives $W\sim N(0,{\sigma }^2)$. Finally, Holder inequality and the residual score bound yieldThis is the residual-score decomposition from square-root Lasso. The normality statement also holds conditionally on a design independent of the noise. If $\gamma$ were chosen from the response noise, for example through the preceding $\gamma (\lambda )$ formula, the direction $R$ would require a separate independence argument; the final calculation treats $\gamma$ as fixed.

The Square-root Lasso estimator with regularization parameter $\gamma >0$ isUnlike the ordinary Lasso, its tuning parameter does not require prior knowledge of the noise standard deviation $\sigma$.

One standard construction uses the Debiased Lasso. Starting from the Square-root Lasso estimate $\hat{\beta }$, estimate a vector ${\hat{\Theta }}_j$ that approximately inverts the $j$th column of the empirical Gram matrix $\hat{\Sigma }=X^{T}X/n$, for example by a Nodewise Lasso. Defineand estimate $\sigma$ by $\hat{\sigma }=\Vert Y-X\hat{\beta }{\Vert }_2/\sqrt{n}$. The approximate two-sided level-$\alpha$ test rejects $H_j^{\beta }$ whenwhere $z_{1-\alpha /2}$ is a standard normal quantile.

Sufficient high-dimensional conditions include a Compatibility condition for the Lasso bounded away from zero, $\gamma \asymp \sqrt{\log p/n}$, $\log p=o(n)$, andtogether with the corresponding sparsity and consistency conditions for the nodewise inverse-Gram estimate. Under these assumptions the Debiased-Lasso asymptotic normality makes the rejection probability under $H_j^{\beta }$ tend to $\alpha$.

The conditional multivariate normal distribution givesFor a jointly Gaussian vector, the residual is independent of the regressor, so $y=z^T\beta +\sigma e$ with $e\sim N(0,1)$ independent of $z$.

The Square-root Lasso minimizesAt a nonzero residual $R$, its KKT condition iswhich proves the required inequality.

Writing the response vector as $Y=f+\sigma e$, the reverse triangle inequality givesThe strong law of large numbers gives $\Vert e{\Vert }_2/\sqrt{n}{\to }_{P}1$, hence $\hat{\sigma }{\to }_P\sigma$.

Under $X\perp \perp Y\mid Z$, Gaussianity makes $e$ independent of $(X,Z)$ and hence of $R$. Conditionally on $R$, $R^{T}e/\Vert R{\Vert }_2\sim N(0,1)$. The remaining numerator term obeysCombining this with $\hat{\sigma }{\to }_P\sigma$ and Slutsky theorem proves the standard-normal limit.

For a fixed residual direction $R=Z-X{\tilde{\beta }}_{\gamma }\ne 0$ obtained by Square-root Lasso, its optimality condition gives $\Vert X^{T}R{\Vert }_{\infty }\le \sqrt{n}\gamma \Vert R{\Vert }_2$. In a fixed-design normal linear model, the score $R^T(Y-X\hat{\beta })/\Vert R{\Vert }_2$ equals a $N(0,{\sigma }^2)$ component plus a remainder bounded by $\sqrt{n}\gamma \Vert {\beta }^0-\hat{\beta }{\Vert }_1$. The direction must be fixed or independent of the response noise for that normality claim.

The subdifferential of the L1 norm consists of vectors $z$ with $z_j=\mathrm{sgn}({\beta }_j)$ when ${\beta }_j\ne 0$ and $z_j\in [-1,1]$ when ${\beta }_j=0$. These coordinate conditions give the Karush-Kuhn-Tucker conditions for Lasso and Square-root Lasso.

A Lasso minimizer with positive residual scale ${\hat{\sigma }}_{\lambda }=\Vert Y-X{\hat{\beta }}_{\lambda }{\Vert }_2/\sqrt{n}$ also minimizes the Square-root Lasso objective at the displayed tuning parameter. Divide its Karush-Kuhn-Tucker conditions by the residual scale and use the subgradient optimality condition.