= L1-penalized logistic regression
{c}
{title2=$\widehat\beta$}
For <independent> binary responses in a <Bernoulli logistic-regression model>, let $\ell(\beta)$ be the summed <log-likelihood function>. With $\lambda>0$, minimize
$$
-\ell(\beta)/n+\lambda\|\beta\|_1.
$$
The <subgradient optimality condition> is
$$
\nabla\ell(\widehat\beta)/n=\lambda z,\qquad
z_j=\operatorname{sgn}(\widehat\beta_j)\ \text{if }\widehat\beta_j\ne0,
\quad z_j\in[-1,1]\ \text{otherwise}.
$$
The sign depends on which <gradient> is named: the <likelihood> <score function> equals $+\lambda z$ after division by $n$, while the gradient of the negative average <log-likelihood function> equals $-\lambda z$. At a zero coordinate the absolute normalized <score function> is at most $\lambda$, explaining exact zero components in the coefficient <vector>.
For a fixed <design matrix> this is a <convex optimization> problem, so the conditions are sufficient as well as necessary. An unpenalized <regression intercept> has its ordinary zero <score function> instead of an L1 condition. If software minimizes the summed negative <log-likelihood function>, the matching penalty is $n\lambda\|\beta\|_1$.
Back to article page
