 Square-root Lasso

The Square-root Lasso minimizesIts residual norm is unsquared. For errors $\epsilon =\sigma Z\ne 0$, the magnitude of the loss gradient at the true regression coefficients is governed byThe unknown noise standard deviation $\sigma$ cancels. This makes a penalty calibrated to that score function pivotal with respect to noise scale.^[ref]

For normalized columns and Gaussian noise, a typical theoretical penalty has order $\sqrt{\log (p)/n}$ without an extra factor of $\sigma$. Prediction guarantees still require assumptions on the number of nonzero regression coefficients, the design matrix and the noise. Pivotal tuning does not remove those assumptions. At an exact zero residual the Euclidean norm is nondifferentiable, so its subdifferential must replace the displayed nonzero-residual gradient.