Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 326 / 3

For a proper convex function $E$ that is lower semicontinuous on a Hilbert space, its proximal operator at scale $\gamma >0$ isThe minimizer is unique because the objective is strongly convex; existence follows from the closed proper convex functional and the coercive quadratic, using an affine lower bound. The subgradient optimality condition is $0\in x-z+\gamma \partial E(x)$, which is exactly the displayed resolvent of a monotone operator relation. Setting $\gamma =1$ gives the unscaled proximity or resolvent operator requested here. This is a nonlinear resolvent of the subdifferential, distinct from the spectral resolvent of a linear operator.

Let $s=Kx$ and assume $s>0$. The scalar Poisson data fidelity integrand has derivative $1-y/s$. Applying the permitted interchange of differentiation and integration in a direction $z$ givesThus the ambient-space Fréchet derivative, or its Hilbert space gradient when the pairing is an inner product, isThe quotient is pointwise, and the adjoint operator transports it back from the data domain to the source domain. The chosen spaces must ensure the integrals and derivative pairing are finite; an output bounded away from zero is a useful sufficient condition.

The printed set of probability density functions and positive cone are not vector spaces, so the notation for a bounded linear map between them is understood as a positive bounded linear operator on ambient function spaces, restricted to admissible inputs. If unit mass is a constraint, admissible directions satisfy ${\int }_{\Omega }z=0$. The displayed gradient is then a representative modulo an additive constant; optimization with that constraint additionally includes its normal cone.

Use the scaled proximal operator ${\mathrm{prox}}_{\gamma E}$; take $\gamma =1$ for each unscaled resolvent.

For the quadratic case, write $D^{T}D=\mathrm{diag}(d_1,\dots ,d_n)$ and $\xi =Qx$. Since $Q$ is an orthogonal matrix, the quadratic proximal term becomes $\Vert \xi -Qz{\Vert }^2/2$. The subgradient optimality condition givesAll $d_j\ge 0$, so each diagonal entry is positive, including when $d_j=0$. Multiplying by the inverse diagonal matrix givesThe proximal operator of Poisson data fidelity separates pointwise. At each $t$, the scalar optimality equation on $x>0$ isSince $y>0$, the two roots have opposite signs. Only the positive root is admissible in the logarithm:The negative root is outside the effective domain. The second derivative $1+\gamma y/x^2$ of the proximal objective is positive, establishing the unique pointwise minimizer. The integral formula is understood on the function spaces where these quantities are admissible.

Finally, for $E(x)=|x|$, positive and negative minimizers obey $x-z+\gamma =0$ and $x-z-\gamma =0$, respectively. At zero the subdifferential condition is $z\in [-\gamma ,\gamma ]$. Combining these cases yields soft thresholding:Thus $\gamma =1$ supplies all three requested closed forms.

For the convex conjugate, use a dual scalar $q$ to avoid confusing it with the fixed centre $z$. Differentiating $qx-\lambda (x-z{)}^2/2$ gives $q-\lambda (x-z)=0$, so the unique maximizing point is $x=z+q/\lambda$. Substitution yieldsThe positivity of $\lambda$ makes the maximized quadratic strictly concave and the convex conjugate finite for every real $q$.

Take $F=(F^{\ast }{)}^{\ast }$ with the usual proper closed convex assumptions, so the Fenchel-Moreau theorem identifies its conjugate with the supplied $F^{\ast }$. Put $z^k=u^k-\tau D^T{v}^k$. The first update is $w^{k+1}={\mathrm{prox}}_{F^{\ast }/\tau }(z^k/\tau )$. By Moreau decomposition, the second update isAlso $\tau (D^T{v}^k+w^{k+1})=u^k-u^{k+1}$, so the argument in the final update contains $u^{k+1}-\tau (D^T{v}^k+w^{k+1})=2u^{k+1}-u^k$. Eliminating the auxiliary variable givesThis is the primal-dual hybrid gradient method with extrapolation parameter one and the primal update performed first. Its saddle point function is $F(u)+⟨Du,v⟩-G(v)$, corresponding to the primal objective $F(u)+G^{\ast }(Du)$.

For proper lower semicontinuous convex functions and a nonempty saddle-point set in these finite-dimensional spaces, the standard convergence of primal-dual hybrid gradient theorem gives the sufficient parameter conditionHere $\Vert D{\Vert }_2$ is the operator norm, or largest singular value. For $D\ne 0$, one possible choice is $\tau =\sigma =\eta /\Vert D{\Vert }_2$ with $0<\eta <1$; for $D=0$ any positive steps satisfy the condition. The existence assumption is necessary: step sizes alone cannot guarantee convergence to a saddle point that does not exist.