Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 326 / 3

For a real Banach space $X$ and a proper convex function $E:X\to \mathbb{R}\cup \{+\infty \}$, the convex conjugate is the extended-valued function on the continuous dual spaceHere proper means that $E$ never takes $-\infty$ and is finite somewhere. The pairing is the duality pairing, or the inner product after identification in a Hilbert space. The conjugate is convex and lower semicontinuous because it is a supremum of continuous affine functions. Its definition gives the Fenchel–Young inequality $E(x)+E^{\ast }(p)\ge ⟨x,p⟩$; the Fenchel-Moreau theorem recovers $E$ as $E^{\ast \ast }$ under the stated closed-convex hypotheses.

For the absolute value function, $px-|x|\le 0$ for all $x$ if $|p|\le 1$, with equality at $x=0$. If $|p|>1$, taking $x=t\mathrm{sign}(p)$ with $t\to \infty$ makes the expression diverge. Thus its convex conjugate isThis is the indicator functional of a constraint set for the dual unit interval, not a zero-one indicator function.

The indicator functional of a constraint set restricts the conjugate supremum to $[-1,1]$. Complete the square:Its maximizer is $x=p$ when $|p|\le 1$, and the nearer endpoint $x=\mathrm{sign}(p)$ otherwise. The convex conjugate of a constrained quadratic is thereforeThis is the unit-threshold Huber loss; both branches and their first derivatives agree at the transition points. The original PDF constrains $|x|\le 1$, including both signs; the damaged TeX loses that absolute value.

In an infinite-dimensional Hilbert space, every bounded sequence has a weakly convergent subsequence, but it need not have a norm-convergent one. For example an orthonormal sequence $e_n$ has pairwise distance $\sqrt{2}$ and therefore no strongly convergent subsequence, although $e_n\rightharpoonup 0$ by Bessel inequality.

More generally, weak sequential compactness of bounded sequences in a reflexive Banach space ensures a weakly convergent subsequence with its limit in that space. Conversely, this property for all bounded sequences characterizes reflexivity, using the Eberlein-Šmulian theorem and weak compactness of the closed unit ball. Thus reflexivity replaces finite-dimensional norm compactness by weak sequential compactness.

In a nonreflexive Banach space there is no such general weak-subsequence guarantee. The coordinate unit vectors in ${\ell }^1$ give an example: by the Schur property, weak convergence would imply norm convergence, impossible for their pairwise distance two. Banach-Alaoglu theorem supplies weak-star compactness of a dual unit ball, which gives convergent subnets, not automatically convergent subsequences. For a separable predual, the weak-star compact dual ball is metrizable and subsequences do suffice. For an arbitrary Banach space one can also regard a bounded sequence in its bidual and extract a weak-star convergent subnet there; the limit need not belong to the original space. These qualifications distinguish weak from weak-star compactness.

On a real Hilbert space, for a proper lower-semicontinuous convex function $E$, the proximal operator isThe quadratic makes this objective strongly convex and coercive; the affine lower bound available for a proper closed convex function and weak lower semicontinuity give existence. The minimizer is unique. The subdifferential optimality condition isFor a positive parameter $t$, ${\mathrm{prox}}_{tE}$ uses $tE(x)$ in the same formula. On a general Banach space, the analogous norm-squared minimization is Banach-space proximal minimization; it can be set-valued and the identity-valued Hilbert derivative must be replaced by the duality mapping.

First use the usual Hilbert space convention for the proximal operator, identifying the dual element $z$ with its representing vector. For $c\ne 0$, completing the square givesSet $v=cx-y$ and multiply the objective by $c^2$. Its variable part becomes $\Vert v-[c(q-z)-y]{\Vert }^2/2+\alpha c^{2}J(v)$. Thus the proximal operator under affine rescaling isNegative $c$ is allowed. If $c=0$ and $J(-y)<\infty$, the penalty is constant and the answer is $q-z$. If $c=0$ and $J(-y)=\infty$, $E$ is identically infinite, so there is no proper proximal problem.

The PDF actually assumes an arbitrary Banach space, so the Hilbert input shift cannot be asserted literally under its hypotheses. Define ${\mathrm{Prox}}_F(s)=\arg {\min }_v\{F(v)+\Vert v-s{\Vert }^2/2\}$ as a possibly empty or set-valued minimizer set. Absolute homogeneity of the norm still gives the valid change of variablesThis retains the linear term, rather than shifting by a dual vector. Equivalently its optimality condition uses the duality mapping $\mathcal{J}=\partial (\Vert \cdot {\Vert }^2/2)$:For a concrete failure of the Hilbert shortcut, take $({\mathbb{R}}^2,\Vert \cdot {\Vert }_1)$, $J=0$, $q=y=0$, $c=1$ and $z=(1,1)$. Every $x_1,x_2\le 0$ with $x_1+x_2=-1$ minimizes $\Vert x{\Vert }_1^2/2+x_1+x_2$ with value $-1/2$. The naive shifted answer $(-1,-1)$ has value zero and is not a minimizer. Hence the simple shifted formula requires Hilbert geometry; general Banach proximal minimizers need not be unique. For $c=0$ with finite constant penalty the Banach condition is $-z\in \mathcal{J}(x-q)$, not generally $x=q-z$.

By the definition of the subdifferential, $p\in \partial J(u)$ meansRearrangement bounds $⟨v,p⟩-J(v)$ by $⟨u,p⟩-J(u)$, and equality is attained at $v=u$. Taking the supremum in the definition of the convex conjugate gives $J^{\ast }(p)=⟨u,p⟩-J(u)$. Conversely this equality bounds every member of that supremum and rearranges to the subgradient inequality. ThereforeApply the same argument to $J^{\ast }$ and use the Fenchel-Moreau theorem $J^{\ast \ast }=J$. With the canonical Hilbert identification of the bidual, the equality is also equivalent to $u\in \partial J^{\ast }(p)$. This proves subgradient inversion under convex conjugacy:The conditions include finiteness at the points in question; expressions involving $+\infty$ are not subgradients merely by formal subtraction.

Let $v={\mathrm{prox}}_J(u)$ and $p=u-v$. The proximal operator optimality condition gives $p\in \partial J(v)$. By subgradient inversion under convex conjugacy, $v\in \partial J^{\ast }(p)$. Since $v=u-p$, this is exactly the optimality condition defining $p={\mathrm{prox}}_{J^{\ast }}(u)$. Uniqueness of both Hilbert proximal minimizers proves Moreau decomposition:Both terms belong to the same Hilbert space after dual identification. No orthogonality of the two terms is asserted for a general convex $J$; that stronger property pertains to special indicator/cone cases.