Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 326 / 2

For a proper convex function $E$ on a real Hilbert space, its subdifferential at $y$ isIn a Banach space, interpret the pairing with an element of the dual space. For a chosen subgradient $p\in \partial E(y)$, the Bregman distance isIf also $q\in \partial E(x)$, its symmetric Bregman distance isThe subgradient inequality proves nonnegativity. These quantities depend on the chosen subgradients and generally are not metrics: they need not distinguish different points or obey the triangle inequality.

For the maximum-entropy regularization functional, differentiation of the scalar integrand gives $(s\log s-s{)}^{\prime }=\log s$. Thus the Fréchet derivative at a positive reference function $y$ is $E^{\prime }(y)=\log y$, and the stated differentiability result gives $\partial E(y)=\{\log y\}$. Consequently,This is the generalized Kullback–Leibler divergence. When both functions integrate to the same mass, the last two terms cancel after integration; for probability density functions it becomes the usual Kullback-Leibler divergence. For two positive functions, adding the reverse distance givesA sufficient setting for the differentiability argument is $L^{\infty }(\Omega )$ with $y$ bounded away from zero. For nonnegative $x$, use $0\log (0/y)=0$ wherever appropriate. Positivity alone in $L^2$ does not automatically provide an open domain on which the Fréchet derivative exists.

For $E(x)=|x|$, the subdifferential at zero is $[-1,1]$. Choose the allowed interior subgradient $p=0$. Its supporting line is horizontal, so the Bregman distance is the vertical gap from $y=0$ to $y=|x|$ at the selected input:More generally, an interior choice $-1<p<1$ gives $D_E^p(1,0)=1-p$. The sketch shows the particular choice $p=0$.

Interpret the normal-operator source condition as the displayed intersection condition: there is $v\in U$ with $w=K^{\ast }Kv\in \partial J(u^{†})$. Fix any $\alpha >0$ and put $\bar{u}=u^{†}+\alpha v$. The subgradient optimality condition for the artificial-data objective isIt holds because its first term equals $-\alpha K^{\ast }Kv=-\alpha w$. By convexity, this is sufficient for global minimality, proving the forward implication. Conversely, if $u^{†}$ minimizes that objective, the same optimality condition givesTaking $v=(\bar{u}-u^{†})/\alpha$ proves the intersection condition. Thus the two conditions are equivalent when the source vector is allowed to be zero.

The printed additional requirement $v\ne 0$ is not equivalent in general. For $K=I$ and $J\equiv 0$, the displayed source condition holds, and choosing $\bar{u}=u^{†}$ makes $u^{†}$ the minimizer, but $\partial J(u^{†})=\{0\}$ forces $v=0$. A nonzero-source version needs an additional hypothesis, such as requiring $w\ne 0$; it cannot be inferred from the printed intersection alone.

Let $w=K^{\ast }Kv\in \partial J(u^{†})$ from the normal-operator source condition, and set $e=u-u^{†}$. Since $u^{†}$ is a least-squares solution, the normal equation for a linear inverse problem gives $K^{\ast }(K{u}^{†}-f)=0$. Expand the objective difference and use the Bregman distance definition:The cross term with the residual of the least-squares solution vanishes by the normal equation; exact data are not required. Compare the minimizing $u_{\alpha }$ with $z=u^{†}-\alpha v$. At $z$, the completed square is zero. Dividing the resulting inequality by $\alpha$ proves the stronger shifted-comparator Bregman boundDropping the nonnegative squared term gives the requested estimate. If the comparator lies outside the effective domain of $J$, its distance is infinite and the inequality is valid but gives no finite error bound.

The relevant nonlinear generalized singular vector relation, without imposing unit-data normalization, isThis is a generalized eigenfunction in the forward-operator metric, rather than a distributional generalized eigenfunction of a linear operator. It supplies the normal-operator source condition with $v=\lambda u^{†}$. For a convex positively one-homogeneous functional, the subgradient inequality applied at zero and at $2u^{†}$ gives $⟨p,u^{†}⟩=J(u^{†})$. If $K{u}^{†}\ne 0$, thenAssume the usual positive eigenvalue $\lambda >0$ and $0<\alpha \lambda \le 1$. By positive homogeneity,The shifted-comparator Bregman bound therefore yieldsThe vector $(1-\alpha \lambda )u^{†}$ is itself a minimizer: $p$ remains a subgradient along the nonnegative ray, and substitution satisfies the subgradient optimality condition. If $K$ is injective, this also proves $u_{\alpha }=(1-\alpha \lambda )u^{†}$. Otherwise distinct minimizers can differ in the kernel while sharing the same predicted data.

For a nonnegative convex positively one-homogeneous functional, the branch continues as $(1-\alpha \lambda {)}_{+}{u}^{†}$ above the threshold, since $p/(\alpha \lambda )\in \partial J(0)$ there. Nonnegativity is needed for that clipping argument; bare positive homogeneity allows signed linear functionals.

Zero Bregman distance does not imply exact recovery. For $K=I$, $J(s)=|s|$, $u^{†}=a>0$ and $0<\alpha <a$, the minimizer is $u_{\alpha }=a-\alpha \ne a$, but $D_J^1(a-\alpha ,a)=0$. Both points touch the same supporting line, as described by zero Bregman distance and supporting faces.