Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 335 / 3

A regularization of an inverse problem replaces the possibly unbounded inverse by a family of continuous reconstruction operators $R_{\alpha }:Y\to X$. For compatible exact data it must recover the minimum-norm solution $x^{†}\in (\ker A{)}^{\perp }$ as $\alpha \downarrow 0$. For noisy data $\Vert y^{\delta }-y\Vert \le \delta$, a parameter rule $\alpha =\alpha (\delta )$ must make both bias and amplified noise tend to zero. The target is the minimum-norm solution because data cannot determine a component in the kernel of a linear map.

Tikhonov regularization minimizesThe quadratic norm penalty makes this functional strictly convex and a coercive function. Its first variation, applied to arbitrary real and imaginary perturbations, givesThe positive operator $A^{\ast }A+\alpha I$ is bounded below by $\alpha I$ and has a bounded inverse. The unique Tikhonov regularization solution isThe Tikhonov stability bound follows from the singular-value gain:Thus reconstruction is stable for each fixed positive $\alpha$. Stability is not uniform as $\alpha \downarrow 0$: that limit restores the unbounded inverse. For exact compatible data,The singular system of a compact operator makes this strong convergence transparent: each positive singular-value coefficient tends to zero and is bounded by the corresponding coefficient of $x^{†}$. Consequentlywhenever $\alpha (\delta )\downarrow 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$, for example $\alpha (\delta )=\delta$. This is a convergent regularization of an inverse problem, not merely a bounded formula.

A singular system of a compact operator consists of positive singular values ${\sigma }_i$ and orthonormal vectors $v_i\in X$, $u_i\in Y$, withThe $v_i$ form an orthonormal basis of $(\ker A{)}^{\perp }$, and the $u_i$ form one of $\overline{\mathrm{ran}A}$. There are finitely many positive singular values for finite-rank $A$; otherwise ${\sigma }_i\to 0$. Obtain the system by applying the spectral theorem for compact self-adjoint operators to $A^{\ast }A$, choosing $A^{\ast }A{v}_i={\sigma }_i^2{v}_i$, and setting $u_i=A{v}_i/{\sigma }_i$. The inner product identities verify orthonormality and the displayed relations.

With inner products linear in the first argument,The normal equation for a linear inverse problem with the Tikhonov regularization penalty acts separately on each $v_i$. Hence the singular-system Tikhonov filter gives the regularized solutionThis norm-convergent series follows from Bessel's inequality and the bound ${\sigma }_i/({\sigma }_i^2+\alpha )\le 1/(2\sqrt{\alpha })$. Data orthogonal to $\overline{\mathrm{ran}A}$ are annihilated by $A^{\ast }$, and the minimizing solution has no component in $\ker A$. The same formula with $y^{\delta }$ reconstructs noisy data. In contrast, the unregularized inverse would divide $(y,u_i)$ by ${\sigma }_i$, with convergence governed by the Picard criterion.

Use the recurrence exactly as printed. Set $B=A^{\ast }A$ and $R=I-B/\alpha$. It givesRepeated substitution yields a function of $A^{\ast }y$ alone:The empty sum is zero for $n=0$. This is Landweber iteration with a Tikhonov initial value. In the singular system of a compact operator, $R{v}_i=r_i{v}_i$, where $r_i=1-{\sigma }_i^2/\alpha$. ThusFor every positive singular value, the finite geometric sum givesTherefore the closed form of a Tikhonov-initialized Landweber iterate isIn the printed notation, $g_{\alpha }(y,u_i)$ means the coefficient $g_{\alpha ,n}({\sigma }_i)(y,u_i)$; its dependence on $n$ and ${\sigma }_i$ is implicit. At $n=0$, the formula reduces to $\sigma /({\sigma }^2+\alpha )$, exactly the Tikhonov regularization initial value. At fixed $n$, its apparent singularity at $\sigma =0$ is removable:Thus each fixed iterate is a bounded reconstruction operator and the series converges in norm.

The terminology in the question requires care. The printed recurrence is explicit Landweber iteration with step $1/\alpha$. Convergence of its nonzero modes requires $|1-{\sigma }_i^2/\alpha |<1$; a sufficient uniform step condition isWith exact compatible data and this condition, $x_n\to x^{†}$. At equality the largest singular-value mode can alternate, and smaller $\alpha$ can produce growing modes. With noisy data, early stopping of Landweber iteration is needed to prevent recovery of arbitrarily unstable small-singular-value components.

Conventional Iterated Tikhonov regularization instead uses the implicit updateWith the same Tikhonov regularization initial value its filter would be $[1-(\alpha /(\alpha +{\sigma }^2){)}^{n+1}]/\sigma$, not the filter derived above. The solution here retains the authoritative printed update rather than silently changing it.