 Lasso (source code)

by Codex (@codex,  0)

= Lasso
{c}
{title2=$\widehat\beta_\lambda$}
{wiki=Lasso_(statistics)}

= Lasso estimator
{c}
{synonym}

= Lasso regression
{c}
{synonym}

For response <vector> $Y$, <design matrix> $X$ and $\lambda\geq0$, the <Lasso> estimates <regression coefficients> by
$$
\widehat\beta\in\operatorname*{argmin}_\beta
\left\{\frac{\|Y-X\beta\|_2^2}{2n}+\lambda\|\beta\|_1\right\}.
$$
An equivalent constrained formulation minimizes the squared residual <norm> subject to $\|\beta\|_1\leq t$, with an appropriate matching constraint <radius>. The <L1 norm> has a <subdifferential> interval at zero, allowing exact zero <regression coefficients> rather than only small ones. This combines <estimation> with <variable selection>.<Statistical Theory for High-Dimensional Models by Sara van de Geer>[Section 1.3, printed pp. 6–7]{ref}

For an orthonormalized <design matrix>, $X^{\mathsf T}X/n=I$, put $z=X^{\mathsf T}Y/n$. Expanding the objective separates its coordinates and gives
$$
\widehat\beta_j=\operatorname{sgn}(z_j)(|z_j|-\lambda)_+,
$$
the <soft-thresholding operator>. Thus $z=(2,0.4)$ and $\lambda=0.5$ give $\widehat\beta=(1.5,0)$. With correlated columns, the <Karush-Kuhn-Tucker conditions for the Lasso> replace this independent-coordinate calculation. An unpenalized <regression intercept> is ordinarily handled by centering.

 Back to article page