Past exam of the mathematics course of the University of Cambridge / 2023 / iii / Paper 339 / 2

The first-order subgradient inequality for a differentiable convex function givesand, after interchanging $v$ and $w$,Adding and rearranging yieldsThus the gradient $F=\nabla f$ is a monotone operator.

If $T(w)=w$, the defining equation becomes $M(w)+F(w)=M(w)$, so $\boxed{F(w)=0}$.

A map is firmly nonexpansive in the $M$-inner product whenPut $p=T(v)$ and $q=T(w)$. The two implicit equations giveBecause $F$ is a monotone operator,Thereforewhich is precisely firm nonexpansiveness. In particular, the preconditioned proximal point algorithm map $T=(M+F{)}^{-1}M$ is nonexpansive in the norm induced by the positive-definite matrix $M$.

Use the sign conventionfor the Lagrangian function in constrained optimization. The Lagrangian dual problem iswhere $f^{\ast }$ is the convex conjugate. For this convex problem with affine equality constraints, the stationarity and feasibility parts of the Karush-Kuhn-Tucker conditions areThey say exactly that the displayed operator satisfiesThus its zeros are precisely the primal-dual optimal points, subject to the usual attainment assumptions.

For $u=(x,z)$ and $\tilde{u}=(y,s)$, the Euclidean inner product givesThe last two terms cancel by the defining property of the matrix transpose, and the first is nonnegative by part a. Hence $F$ is a monotone operator.

The assertion uses the positive parameters required for $M$ to be a preconditioner: assume $\alpha >0$ and $\beta >0$. For $x\in {\mathbb{R}}^n$ and $z\in {\mathbb{R}}^m$, complete the square:The matrix 2-norm bound $\Vert Ax{\Vert }_2\le \Vert A{\Vert }_2\Vert x{\Vert }_2$ shows thatfor $x\ne 0$ when $\alpha \beta >\Vert A{\Vert }_2^2$. If $x=0$ and $z\ne 0$, the square contributes $\beta \Vert z{\Vert }_2^2>0$. Thus $\boxed{M\text{ is positive definite}}$. Equivalently, the Schur complement of the lower-right block is $\alpha I-A^{T}A/\beta \succ 0$.

Taken literally without the positivity inherited from part b, the product condition alone is insufficient: $A=0$ and $\alpha =\beta =-1$ is a counterexample. Thus $\alpha ,\beta >0$ is a necessary implicit hypothesis.

Write $w_k=(x_k,z_k)$ and $w_{k+1}=(x_{k+1},z_{k+1})$. ExpandinggivesThe off-diagonal terms in the first equation cancel. By the optimality condition for the proximal operator,The second equation then becomes the explicit linear updateThus each step of this preconditioned proximal point algorithm uses only one evaluation of the proximal operator of ${\alpha }^{-1}f$, together with applications of the linear map $A$ and its transpose $A^T$.