Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 210 / 3

Each row of the oriented incidence matrix records one signed endpoint difference, henceReversing an edge changes the sign of its row, and permuting the edges permutes rows; neither operation changes the sum of absolute differences. Thus the graph total variation is independent of the chosen orientation and ordering. If $D\theta =0$, values agree at the endpoints of every edge. Any two graph vertices of a connected graph are joined by a graph path, so their values agree. Therefore $\ker D=\mathrm{span}\{1\}$.

Let $h=\hat{\theta }-{\theta }^{\ast }$. The defining minimum for graph total variation denoising, compared with the feasible value ${\theta }^{\ast }$, gives $n^{-1}\Vert z-h{\Vert }_2^2+\lambda \Vert D\hat{\theta }{\Vert }_1\le n^{-1}\Vert z{\Vert }_2^2+\lambda \Vert D{\theta }^{\ast }{\Vert }_1$. Expand the squared Euclidean norm and cancel $\Vert z{\Vert }_2^2/n$ to obtain the basic inequality for a penalized least-squares estimator:

Decompose $h={\Pi }_{1}h+D^{†}Dh$ using the incidence pseudoinverse decomposition. The Cauchy-Schwarz inequality controls the constant component, and the duality of the ${\ell }_{\infty }$ and ${\ell }_1$ norms controls the remaining component:On the stipulated event, the second term contributes at most $\lambda \Vert Dh{\Vert }_1$ after multiplication by $2/n$. The triangle inequality gives $\Vert Dh{\Vert }_1\le \Vert D\hat{\theta }{\Vert }_1+\Vert D{\theta }^{\ast }{\Vert }_1$, so the negative penalty in the basic inequality for a penalized least-squares estimator cancels. We obtainon an event of probability at least $1-\delta$.

With the usual variance-proxy convention for a sub-Gaussian random vector, every column inner product is a sub-Gaussian random variable with proxy at most ${\tau }^2$. The two-sided Chernoff bound and union bound implywith probability at least $1-\delta$. No independence between these inner products is required.

The printed coefficient-one bound is false under this convention. Take $k=m=1$, $A=[1]$, and $g=\pm \tau$ with equal probability. This Rademacher random variable has the stipulated variance proxy, but for $\delta =0.99$ the printed threshold is less than $\tau$, whereas $|g|=\tau$ surely. Even a general sub-Gaussian assumption does not imply Gaussian concentration inequality for arbitrary Lipschitz functions. Under the stronger convention $\mathbb{E}{e}^{⟨a,g⟩}\le e^{{\tau }^2\Vert a{\Vert }_2^2/4}$, the same union bound proves the printed constants. That convention, however, differs from the binomial variance proxy used in 1(b).

Let $B_{\delta }=\sqrt{\log (2m)}+\sqrt{\log (1/\delta )}$. The columns of $D^{†}/L$ have Euclidean norm at most one. Under the usual sub-Gaussian random vector convention, 3(d) justifies the tuning choice $\lambda =2\sqrt{2}L{B}_{\delta }/n$, which is independent of ${\theta }^{\ast }$. Also $\Vert {\Pi }_{1}z{\Vert }_2=|⟨z,1/\sqrt{n}⟩|$, so the Chernoff bound gives $\Vert {\Pi }_{1}z{\Vert }_2^2\le c_{\delta }=2\log (2/\delta )$ with probability at least $1-\delta$.

On the intersection of these events, of probability at least $1-2\delta$, set $a=\Vert h{\Vert }_2/\sqrt{n}$ and $b=\Vert {\Pi }_{1}z{\Vert }_2/\sqrt{n}$. The weighted Young inequality gives $2ab\le t^2{a}^2+b^2/t^2$ for $0<t<1$. Applying it to 3(c) and rearranging yields the rigorous boundIf the coefficient-one concentration statement in 3(d) is taken as an extra hypothesis, instead choose $\lambda =2L{B}_{\delta }/n$; the identical argument gives precisely the printed coefficient $4$ in the first term. Thus the deterministic argument and its risk rate are sound; its concentration constants depend on correcting or strengthening 3(d).