Active hinge set 2026-10-06
For a sample term , the active hinge set contains indices with . At strict negative margins the hinge is inactive, while zero margins allow fractional hinge-loss optimality weights. Away from zero margins, the active set stays fixed on a sufficiently small candidate solution neighborhood.
Write the objective as with . Each hinge loss is finite and convex, so the subdifferential sum rule applies without a qualification gap. Its subdifferential is
This formula also covers , if such a sample is allowed, because the loss is then constant. For a convex function, zero in the subdifferential is necessary and sufficient for global optimality. Thus is optimal exactly when there are weights such that
Equivalently, using and ,
The notation denotes the segment between and , also when . This avoids reversing interval endpoints for negative-class samples.
Because , the objective is strongly convex, and makes it coercive. It is continuous, so a minimizer exists and is unique for every finite data vector . The weights at exact margins need not be unique even though is. These are the hinge-loss optimality weights for the one-dimensional support vector machine.
Yes: the solution map is locally affine, hence has the Aubin property. Keep the regularization parameter fixed and define the active hinge set
Since none of the margins vanishes, the optimality condition at the reference data has no fractional weights:
For nearby , define the candidate
Each map is continuous and is nonzero at . Because there are finitely many samples, all their signs remain unchanged on a common neighborhood of . Therefore the same set is active at for every . The hinge-loss optimality weights are still on and off , and the defining equation for proves the subgradient optimality condition. Strong convexity makes this candidate the unique global minimizer. Consequently the solution map of a parametric optimization problem satisfies
This argument establishes stability of the active pattern without presupposing continuity of the unknown optimizer; continuity is used only for an explicit candidate and then optimality is checked.
For , the Cauchy-Schwarz inequality gives
This is the Aubin property with . In fact this constant is the exact local Lipschitz continuity modulus of the affine branch, since its derivative is
The active-set dependence of this derivative is the active-set sensitivity of hinge-loss minimization. At a vanishing margin the affine-branch argument no longer applies, although failure of this particular argument does not by itself prove failure of the Aubin property.
Figure 1.
For two scalar samples with alpha equal to one and data (s,1), the optimal coefficient follows two locally affine branches separated by a branch pinned to an exact hinge margin
. Details.
For the illustration, take , , and with . The hinge-loss optimality weights give
On the middle branch the first margin is exactly zero, so that branch lies outside the strict-margin hypothesis. The example makes clear why the local affine conclusion is tied to that hypothesis, and also shows that a zero margin need not destroy local Lipschitz stability.