L1-penalized logistic regression
ID: l1-penalized-logistic-regression
For independent binary responses in a Bernoulli logistic-regression model, let be the summed log-likelihood function. With , minimizeThe subgradient optimality condition isThe sign depends on which gradient is named: the likelihood score function equals after division by , while the gradient of the negative average log-likelihood function equals . At a zero coordinate the absolute normalized score function is at most , explaining exact zero components in the coefficient vector.
For a fixed design matrix this is a convex optimization problem, so the conditions are sufficient as well as necessary. An unpenalized regression intercept has its ordinary zero score function instead of an L1 condition. If software minimizes the summed negative log-likelihood function, the matching penalty is .
New to topics? Read the docs here!