Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 335 / 3

Let $({\sigma }_j,u_j,v_j)$ be a singular system of a compact operator $A:X\to Y$, soThe Moore–Penrose inverse of an operator has domainand acts bywith the orthogonal component of $y$ sent to zero. Equivalently, its domain consists of the data satisfying the Picard criterion. It obeys

If $Ax=y$ is exactly solvable, every solution is $x^{†}+z$ with $z\in \ker A$, andis the unique minimum-norm least-squares solution. It recovers the component of the original $x$ orthogonal to the null space; no data can determine the null-space component.

The normal equation for a linear inverse problem isLandweber iteration is the stationary iterationA sufficient step-size condition isIf $y\in \mathcal{D}(A^{†})$ and $x_0\in (\ker A{)}^{\perp }$, thenFor an arbitrary initial iterate, its null-space component is unchanged and the limit is

Forthe Frechet derivative in direction $h$ isThus the Hilbert-space gradient isThe gradient descent update with step size $\gamma$ is consequentlywhich is exactly Landweber iteration. Its stationary points solve the normal equation and minimize the convex quadratic functional $J$.

SetStarting from $x_0=0$, repeated substitution in Landweber iteration givesThis is the finite partial sum of a Neumann series. Whenever $A^{\ast }A$ is boundedly invertible on the relevant subspace and $\Vert I-\gamma A^{\ast }A\Vert <1$,so

For a genuinely compact operator on an infinite-dimensional space, nonzero singular values can accumulate at zero, so this inverse is generally unbounded and the Neumann series need not converge in operator norm. Under the condition in part ii it nevertheless converges componentwise on admissible data to the Moore–Penrose inverse of an operator; stopping after finitely many terms suppresses poorly determined small-singular-value components and acts as a regularization of an inverse problem.