Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 326 / 1

For a bounded linear operator $K:U\to V$ between Hilbert spaces, the Moore–Penrose inverse of an operator assigns to each admissible datum its unique minimum-norm least-squares solution. Its domain isWrite $f=Ku+h$, where $u\in (\ker K{)}^{\perp }$ and $h\in (\mathrm{ran}K{)}^{\perp }=\ker K^{\ast }$. Then $K^{†}f=u$. Restricting to $(\ker K{)}^{\perp }$ removes the ambiguity among solutions differing by an element of the kernel. The orthogonal decomposition also shows that $K{K}^{†}f$ is the orthogonal projection of $f$ onto $\overline{\mathrm{ran}K}$, and $K^{†}Ku$ is the orthogonal projection of $u$ onto $(\ker K{)}^{\perp }$. The inverse is defined on a dense domain and need not be a bounded linear operator when the range is not closed.

The given map is the unilateral shift operator. Its adjoint operator is the left shift operator, sinceThe shift is an isometry with trivial kernel and closed range $\{f:f_1=0\}$. Its range has orthogonal complement $\mathrm{span}\{e_1\}$, so every square-summable datum is admissible. Consequently,Indeed $K^{†}K=I$, while $K{K}^{†}f=f-f_1{e}_1$ is the required orthogonal projection. The discarded first component is the residual of the least-squares solution, not an obstruction to the existence of the Moore–Penrose inverse of an operator.

Choose a singular system of a compact operator with $K{v}_j={\sigma }_j{u}_j$, $K^{\ast }{u}_j={\sigma }_j{v}_j$ and ${\sigma }_j>0$. The Picard criterion, together with range orthogonality, givesFor necessity, expand a solution's component in $(\ker K{)}^{\perp }$ in the orthonormal vectors $v_j$; its coefficients must be $⟨f,u_j⟩/{\sigma }_j$. Their square sum is finite by Bessel's inequality. Conversely, the displayed square sum defines a convergent Hilbert space seriesApplying the bounded linear operator $K$ to the partial sums yields the expansion of $f$ in $\overline{\mathrm{ran}K}=(\ker K^{\ast }{)}^{\perp }$. Thus its limit solves $Ku=f$. The perpendicularity condition is essential here: the Picard criterion alone characterizes the domain of the generalized inverse, which also includes components in $\ker K^{\ast }$.

A linear regularization is a family of bounded linear operators $R_{\alpha }:V\to U$, $\alpha >0$, for which $R_{\alpha }f\to K^{†}f$ for every $f\in \mathcal{D}(K^{†})$ as $\alpha \downarrow 0$. To obtain a convergent regularization of an inverse problem for noisy data, choose a regularization parameter so that the exact-data approximation error and the amplified noise both vanish.

An example is Tikhonov regularization:The positive quadratic term makes $K^{\ast }K+\alpha I$ invertible. The Tikhonov filter norm bound follows from the filter $s/(s^2+\alpha )$, whose maximum for $s\ge 0$ is $1/(2\sqrt{\alpha })$. Therefore $\Vert R_{\alpha }\Vert \le 1/(2\sqrt{\alpha })$. The spectral factors $s^2/(s^2+\alpha )$ tend to one on the positive spectrum; dominated convergence theorem gives consistency on $\mathcal{D}(K^{†})$. The noise-bias decomposition for linear regularization then proves convergence whenever $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$, for example $\alpha (\delta )=\delta$ for sufficiently small positive noise levels.

The Volterra integration operator is injective and has dense range. Its range of the Volterra integration operator is $\{f\in H^1(0,1):f(0)=0\}$, and its Moore–Penrose inverse of an operator is differentiation on this domain. Thus the error estimate for $K^{†}f$ implicitly requires $f(0)=0$, in addition to the printed $H^2$ assumption. For an arbitrary $H^2$ function the same calculation bounds error against $f^{\prime }$, but that function need not be in $\mathcal{D}(K^{†})$.

Put $a=(1-\alpha )/2$, $b=(1+\alpha )/2$ and $h=\alpha /2$. The three stencils are forward, central finite differences, and backward on $[0,a)$, $[a,b)$ and $[b,1]$, respectively. All shifted evaluation points lie in $[0,1]$. For $g\in L^2(0,1)$, apply $|A-B{|}^2\le 2(|A{|}^2+|B{|}^2)$ and change variables in each of the six evaluation terms. The resulting integration intervals arePair $[0,a]$ with $[a,1-\alpha ]$, $[\alpha ,b]$ with $[b,1]$, and the two middle intervals. Each pair has disjoint interiors, so almost every point is counted at most three times. This overlap multiplicity bound for piecewise difference operators provesFor the bias on the middle interval, write the central finite difference as an average of $f^{\prime }$. The fundamental theorem of calculus giveswhere $\Vert k_{\alpha }{\Vert }_1=\alpha /4$. Extend $f^{\prime \prime }$ by zero to the real line. Young's convolution inequality bounds the middle-interval $L^2$ error by $\alpha c/4$. Combining its squared error with the supplied outer-interval estimate gives the central-difference noise-bias boundThe triangle inequality and the operator norm estimate now giveFor $c>0$, balancing noise and approximation bias minimizes this upper bound atfor sufficiently small $\delta$ that $\alpha <1/2$. Capping the parameter at $1/4$ supplies an admissible rule for all positive noise levels. More generally, $\alpha \to 0$ and $\delta /\alpha \to 0$ suffice; $\alpha =\sqrt{\delta }$ avoids needing $c$ or dividing by $c=0$.

To prove convergence for every admissible datum, not just $H^2$ data, let $f=Ku$ with $u\in L^2(0,1)$ and extend $u$ by zero. Each stencil is a forward, centred or backward average of $u$. Translation continuity in Lp makes each averaged difference from $u$ tend to zero in $L^2(\mathbb{R})$. Restricting those three bounds to their respective intervals proves $R_{\alpha }Ku\to u$. The noise-bias decomposition for linear regularization, with $\delta /\alpha \to 0$, finishes the convergent regularization of an inverse problem.