Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 76 / 3

For noisy data, exact equality $Ax=y$ may be impossible: the data can have a component outside the operator range. A least-squares solution of a linear inverse problem instead minimizesFor any variation $h\in X$, the adjoint operator givesThus statistical homogeneity in every direction is the normal equation for a linear inverse problem, $A^{\ast }Ax=A^{\ast }y$. It is also sufficient for a global minimum: when it holds,The residual is orthogonal to the closure of the range, since it lies in $\ker A^{\ast }$. A normal-equation solution therefore fits the component of the data representable by the forward map, while leaving the perpendicular component unfitted. If there is more than one such solution, select the minimum-norm least-squares solution, which lies in $(\ker A{)}^{\perp }$.

These facts explain the formulation, but do not make the inverse problem well posed. A compact operator of infinite rank typically has nonclosed operator range, so even the least-squares infimum need not be attained. A least-squares solution exists exactly when the orthogonal projection of $y$ onto $\overline{\mathrm{Ran}A}$ belongs to $\mathrm{Ran}A$. In a singular system of a compact operator, this becomes the Picard criterion. Also, the nonzero eigenvalues of $A^{\ast }A$ are the squares of the singular values of $A$: solving the operator normal equation directly can worsen numerical conditioning. The operator normal equation expresses least squares; it does not itself regularize the inverse problem. The subsequent finite Landweber iteration and its stopping rule provide the inverse-problem regularization.

Choose a fixed positive Landweber relaxation parameter $\tau$ and rearrange the normal equation for a linear inverse problem as the fixed point equationSuccessive substitution gives Landweber iterationSet $B=A^{\ast }A$ and $P=I-\tau B$. Induction yields the closed polynomial-operator expressionFor $n=0$ the sum is empty, giving zero. The expression is well defined without an inverse of $A^{\ast }A$, which can have a kernel of a linear map and an unbounded inverse on its range. Each iterate lies in the closure of $\mathrm{Ran}{A}^{\ast }$, so initialization at zero eliminates the arbitrary null-space component of a solution. The unrelaxed choice $\tau =1$ gives $x_{n+1}=(I-A^{\ast }A)x_n+A^{\ast }y$ and the corresponding sum with $\tau =1$; it needs the convergence restriction in part (d).

Use the convention required by the printed expansion:where $u_i\in Y$ and $v_i\in X$ form the positive-singular-value parts of the singular system of a compact operator. Consequently $A^{\ast }A{v}_i={\sigma }_i^2{v}_i$. Apply part (b)'s polynomial to each such eigenvector, using noisy data:The geometric series then gives the full inverse coefficient, or Landweber full inverse filter,In particular, for the unrelaxed iteration $\tau =1$, $g_{\alpha }(\sigma )=[1-(1-{\sigma }^2{)}^{1/\alpha }]/\sigma$. The dimensionless Landweber spectral filter is instead $\sigma g_{\alpha }(\sigma )$; the printed expression uses the full coefficient and therefore needs the denominator $\sigma$.

For each fixed finite $n$, the filter extends continuously at zero by $g_{1/n}(0)=0$, since $g_{1/n}(\sigma )=n\tau \sigma +O({\sigma }^3)$ as $\sigma \to 0$. Small singular values are suppressed rather than immediately divided into noisy data. A component of $y^{(\delta )}$ in $\ker A^{\ast }$ contributes nothing. The parameter here is discrete: $1/\alpha$ is an integer. This matters when $1-\tau {\sigma }^2<0$; arbitrary real powers are not intended. A family for every $\alpha >0$ can use $n=\lfloor 1/\alpha \rfloor$ instead.

For nonzero $A$, a sufficient step-size condition for strong convergence of relaxed Landweber iteration isFor the unrelaxed iteration, this reads $\Vert A{\Vert }^2<2$. The data must also admit a minimum-norm least-squares solution $x^{†}$; for the exact problem, $y\in \mathrm{Ran}A$ is sufficient. More generally, the projected-data condition in part (a), or the Picard criterion ${\sum }_i|⟨y,u_i⟩{|}^2/{\sigma }_i^2<\infty$, ensures the required solution.

Every positive singular value satisfies $|1-\tau {\sigma }_i^2|<1$, so for fixed ${\sigma }_i>0$,This is the correct inverse coefficient. For exact data with a least-squares solution, the norm error isThe summable majorant is precisely the Picard criterion, so the dominated convergence theorem proves strong convergence, despite the absence of a uniform contraction constant when the singular values approach zero. Initialization at zero selects the solution orthogonal to $\ker A$. At $\tau =2/\Vert A{\Vert }^2$, a top singular mode of a nonzero compact operator has multiplier $-1$ and can oscillate indefinitely; the strict step bound avoids this failure.

Pointwise convergence of filters alone is not a full noisy-data inverse-problem regularization proof. For each finite $n$, put $t=\tau {\sigma }^2$; the step restriction gives $|1-t|\le 1$, andThis proves the uniform noise bound for relaxed Landweber iterationTogether with the exact-data convergence,whenever $n=n(\delta )\to \infty$ and $\delta \sqrt{n(\delta )}\to 0$. Equivalently, the a priori regularization parameter choice satisfies $\alpha (\delta )\to 0$ and $\delta /\sqrt{\alpha (\delta )}\to 0$. For example, $n(\delta )=\lfloor {\delta }^{-1}\rfloor$ works as $\delta \to 0$. The sharper familiar bound $\delta \sqrt{n\tau }$ is available when $\tau \Vert A{\Vert }^2\le 1$.

Thus finite iteration followed by an appropriate stopping rule is a convergent regularization of an inverse problem. Taking $n\to \infty$ for fixed noisy data is not the same operation and can amplify the small-singular-value noise without bound. If $A=0$, the zero starting iterate stays zero and is the minimum-norm least-squares solution for every datum; the step restriction is needed only for nonzero $A$.