Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 325 / 2 / vi

Set $r^k=A{x}^k-c$. Completing the square in the $y$-minimization givesThus, with $\sigma =\tau /(1+2\tau )$, the steps in terms of the proximal map of $\varphi$ areThe factor $2$ comes from the squared residual without a $1/2$ prefactor. The proximal map can equivalently be denoted $(I+\sigma \partial \varphi {)}^{-1}$. Since $\varphi$ is a finite convex function, it is continuous, and the quadratic makes the proximal minimizer unique.

A general convex $\varphi$ may be nondifferentiable, so this last example need not satisfy the full $L$-smooth hypothesis of the preceding part. The formulas nevertheless remain well-defined: only the $x$ derivative is used explicitly. If $\varphi$ has an $L_{\varphi }$-Lipschitz gradient, one may take $L=2(\Vert A{\Vert }_2^2+1)+L_{\varphi }$ in the smooth proof, where $\Vert A{\Vert }_2$ is the operator norm.

There is also a useful nonsmooth extension. Put $h(x,y)=\Vert Ax+y-c{\Vert }_2^2$, whose Lipschitz gradient constant is $L_h=2(\Vert A{\Vert }_2^2+1)$. For two updates, retain the mixed-point differences $(p,q)$ above, let $(a,b)=\nabla h(w)-\nabla h(\tilde{w})$, and let $d$ be the difference of the two selected subgradients of $\varphi$ at $y^{+},{\tilde{y}}^{+}$. The input difference is $(p,q+\tau (b+d))$ and the output difference is $(p-\tau a,q)$. Monotonicity of $\partial \varphi$ gives $⟨q,d⟩\ge 0$. The same norm expansion now yieldsFor $0<\tau L_h\le 1$, discard the nonpositive $b$ term and use $2\tau /L_h-{\tau }^2\ge {\tau }^2$. The result is the firmly nonexpansive mapping inequality for the full residual $\tau (a,b+d)$. Fixed points satisfy ${\nabla }_{x}h=0$ and $0\in {\nabla }_{y}h+\partial \varphi$, exactly the subgradient optimality condition for the convex objective. Thus the stated algorithm also converges for nonsmooth $\varphi$ under this sufficient step bound, provided a minimizer exists; this extension uses monotonicity, not an unavailable gradient of $\varphi$.