Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 325 / 3

Let $S:{\mathbb{R}}^p\rightrightarrows {\mathbb{R}}^q$ be a set-valued mapping and let $\bar{v}\in S(\bar{u})$. The Aubin property at $(\bar{u},\bar{v})$ means there are neighborhoods $U$ of $\bar{u}$, $V$ of $\bar{v}$, and a finite constant $\kappa \ge 0$ such thatHere ${\mathbb{B}}_q$ is the closed unit ball. Equivalently, every solution $v\in S(u)$ near $\bar{v}$ can be matched to a solution $v^{\prime }\in S(u^{\prime })$ within $\kappa \Vert u-u^{\prime }{\Vert }_2$. The localization is on the left-hand side: the matching point need not be in $V$. The property is also called the Lipschitz-like property.

For sensitivity analysis, let $u$ represent data or perturbations and $S(u)$ the set of feasible or optimal solutions. The Aubin property bounds how far a nearby solution can move when the data change. Taking $u=\bar{u}$, $v=\bar{v}$ also guarantees a nearby solution for each sufficiently small perturbation $u^{\prime }$. It is a stability estimate for a relation, and by itself does not imply uniqueness or differentiability. For a single-valued solution map, it reduces to local Lipschitz continuity.

Work in finite-dimensional Euclidean spaces. Suppose the graph of a set-valued mapping,is locally closed at $(\bar{u},\bar{v})\in \mathrm{gph}S$. Local closedness means that its intersection with some neighborhood of this point is closed relative to that neighborhood. This assumption is part of the criterion.

For a set $C$ and $z\in C$, define the Fréchet normal cone byAt an isolated point the condition is vacuous, so all vectors are regular normals. The limiting normal cone consists of limits of these regular normals at nearby points:It can be nonconvex. The Mordukhovich coderivative isThe minus sign in the output-dual component is part of the definition. For a single-valued continuously differentiable mapping $s$, the graph normal is $(Ds(\bar{u}{)}^T{v}^{\ast },-v^{\ast })$, giving $D^{\ast }s(\bar{u}\mid s(\bar{u}))(v^{\ast })=\{Ds(\bar{u}{)}^T{v}^{\ast }\}$. Thus the coderivative extends a transpose derivative, rather than the ordinary forward derivative.

The exact Mordukhovich criterion isThis is equivalent to excluding a nonzero horizontal limiting graph normal $(u^{\ast },0)$. One must use the limiting normal cone; simply checking regular normals at the reference point can miss normals inherited from neighboring graph pieces.

For sensitivity analysis, apply this test to a solution map or use coderivative and normal cone calculus to express its graph normals through optimality constraints. Triviality of this kernel then proves Lipschitz-like stability without solving the perturbed problem explicitly. Under these same finite-dimensional, locally closed assumptions, the exact Lipschitz modulus is the outer normThe zero-kernel condition is the qualitative criterion; the modulus quantifies the sensitivity bound. For the next problem an explicit local formula is simpler than evaluating these graph normals.

Write the objective as $F_a(x)=\alpha x^2/2+n^{-1}{\sum }_i{\ell }_i(x)$ with ${\ell }_i(x)=\max \{0,1-a_{i}x\}$. Each hinge loss is finite and convex, so the subdifferential sum rule applies without a qualification gap. Its subdifferential isThis formula also covers $a_i=0$, if such a sample is allowed, because the loss is then constant. For a convex function, zero in the subdifferential is necessary and sufficient for global optimality. Thus $x$ is optimal exactly when there are weights ${\lambda }_i$ such thatEquivalently, using $I(x)=\{i:a_{i}x<1\}$ and $J(x)=\{i:a_{i}x=1\}$,The notation $[0,1]a_i$ denotes the segment between $0$ and $a_i$, also when $a_i<0$. This avoids reversing interval endpoints for negative-class samples.

Because $\alpha >0$, the objective is strongly convex, and $F_a(x)\ge \alpha x^2/2$ makes it coercive. It is continuous, so a minimizer exists and is unique for every finite data vector $a$. The weights at exact margins need not be unique even though $x$ is. These are the hinge-loss optimality weights for the one-dimensional support vector machine.

Yes: the solution map is locally affine, hence has the Aubin property. Keep the regularization parameter $\alpha >0$ fixed and define the active hinge setSince none of the margins vanishes, the optimality condition at the reference data has no fractional weights:For nearby $a$, define the candidateEach map $a\mapsto 1-a_i{x}_I(a)$ is continuous and is nonzero at $\hat{a}$. Because there are finitely many samples, all their signs remain unchanged on a common neighborhood $U$ of $\hat{a}$. Therefore the same set $I$ is active at $x_I(a)$ for every $a\in U$. The hinge-loss optimality weights are still $1$ on $I$ and $0$ off $I$, and the defining equation for $x_I(a)$ proves the subgradient optimality condition. Strong convexity makes this candidate the unique global minimizer. Consequently the solution map of a parametric optimization problem satisfiesThis argument establishes stability of the active pattern without presupposing continuity of the unknown optimizer; continuity is used only for an explicit candidate and then optimality is checked.

For $a,b\in U$, the Cauchy-Schwarz inequality givesThis is the Aubin property with $\kappa =\sqrt{|I|}/(\alpha n)$. In fact this constant is the exact local Lipschitz continuity modulus of the affine branch, since its derivative isThe active-set dependence of this derivative is the active-set sensitivity of hinge-loss minimization. At a vanishing margin the affine-branch argument no longer applies, although failure of this particular argument does not by itself prove failure of the Aubin property.

For the illustration, take $n=2$, $\alpha =1$, and $a=(s,1)$ with $s>0$. The hinge-loss optimality weights giveOn the middle branch the first margin is exactly zero, so that branch lies outside the strict-margin hypothesis. The example makes clear why the local affine conclusion is tied to that hypothesis, and also shows that a zero margin need not destroy local Lipschitz stability.