Hinge loss

The hinge loss is $\max (0,1-yh(x))$ and is one-Lipschitz in the signed margin.

For a sample term $\max \{0,1-a_{i}x\}$, the active hinge set contains indices with $1-a_{i}x>0$. At strict negative margins the hinge is inactive, while zero margins allow fractional hinge-loss optimality weights. Away from zero margins, the active set stays fixed on a sufficiently small candidate solution neighborhood.

With fixed positive quadratic regularization, if all reference hinge margins are nonzero, the one-dimensional minimizer has local formula $x(a)=(\alpha n{)}^{-1}{\sum }_{i\in I}{a}_i$. The active hinge set is verified using continuity of this explicit candidate, then strong convexity identifies it as the optimizer. Its exact Euclidean Lipschitz modulus is $\sqrt{|I|}/(\alpha n)$.

The subgradient of $\max \{0,1-a_{i}x\}$ is $-{\lambda }_i{a}_i$, with ${\lambda }_i=1$ below the margin, ${\lambda }_i=0$ beyond it and ${\lambda }_i\in [0,1]$ at equality. For quadratic regularization the one-dimensional optimality condition is $\alpha nx={\sum }_i{\lambda }_i{a}_i$. The optimizer can be unique even if the weights at equality are not.