Past exam of the mathematics course of the University of Cambridge / 2022 / iii / Paper 326 (Inverse Problems)

For a bounded operator $A:X\to Y$ between Hilbert spaces, the Moore--Penrose inverse of a Hilbert-space operator is defined onand has range $\mathcal{R}(A^{†})=\mathcal{N}(A{)}^{\perp }$. Its Penrose equations areEquivalently, $A^{†}A=P_{\mathcal{N}(A{)}^{\perp }}$ and $A{A}^{†}$ is the restriction of $P_{\overline{\mathcal{R}(A)}}$ to $\mathcal{D}(A^{†})$.

The statement is true without an additional rank assumption. If $A=U\Sigma V^H$ is a singular value decomposition, then

This statement is also always true. The same singular value decomposition gives

This statement is true. The matrix $A^{†}A$ is the orthogonal projection onto the row space $\mathcal{R}(A^H)$, so its eigenvalues are zero and one. Consequently

The Reverse-order law for the Moore--Penrose inverse is false in general. TakeThenA sufficient condition isso that $A$ has full column rank and $B$ has full row rank. Indeed $A^{†}A=I_n$ and $B{B}^{†}=I_n$; these identities make $B^{†}{A}^{†}$ satisfy all four Penrose equations for $AB$.

If $B={\sup }_j|d_j|<\infty$, then for every $f\in {\ell }^2$,so the diagonal operator on sequence space is bounded. Conversely, for the standard unit vector $e_j$,so boundedness of $D$ implies ${\sup }_j|d_j|\le \Vert D\Vert <\infty$.

Part i gives $\Vert D\Vert \le B$. Conversely, choose indices $j_k$ with $|d_{j_k}|\to B$. Since $\Vert e_{j_k}{\Vert }_2=1$,Taking the limit proves $\boxed{\Vert D\Vert =B}$, whether or not the supremum is attained.

The Moore-Penrose inverse acts coordinatewise ason the domainThe coordinates supported where $d_j=0$ form $\mathcal{R}(D{)}^{\perp }$ and are sent to zero. The inverse is continuous exactly when the nonzero diagonal entries are bounded away from zero,apart from the trivial all-zero operator, whose inverse is zero. Otherwise unit vectors along entries tending to zero show that $D^{†}$ is unbounded.

For $d_j=1/j$, the equation $Du=f$ forces $u_j=j{f}_j$. It has a solution in ${\ell }^2$ exactly whenFor example, $f_j=1/j$ defines an element of ${\ell }^2$, but its forced preimage is the constant sequence $u_j=1$, which is not in ${\ell }^2$. Thus existence can fail. Since every $d_j$ is nonzero, $D$ has trivial kernel and any solution is unique.

The proposed stability property is false. Set $u^{\ast }=0$, $f^{\ast }=0$, and take $u_n=e_n$. Thenwhile $\Vert u_n-u^{\ast }{\Vert }_2=1$ for every $n$. Hence the inverse is not continuous on its range and the recovery problem is not stable with respect to perturbations. This is the standard unbounded inverse on a nonclosed operator range.

The extended-real functional $f:X\to \overline{\mathbb{R}}$ is sequentially lower semicontinuous in the topology ${\tau }_X$ when every sequence $x_n\to x$ in ${\tau }_X$ satisfies

The assertion is understood for every $\eta \in \mathbb{R}$ and for a topology in which sequentially closed sets are closed, in particular the norm topology of a Banach space. Suppose first that $f$ is sequentially lower semicontinuous and $x_n\in {\Omega }_{\eta }=\{x:f(x)\le \eta \}$ converges to $x$. Thenso $x\in {\Omega }_{\eta }$ and the sublevel set is closed. Conversely, if all sublevel sets are closed but lower semicontinuity fails, there are $x_n\to x$ and a real $\eta <f(x)$ with a subsequence satisfying $f(x_{n_k})\le \eta$. Closedness of ${\Omega }_{\eta }$ would put $x$ in that set, a contradiction. This proves the closed-sublevel-set characterization of lower semicontinuity.

Use the extended-real characteristic functionalFor $\eta <0$ its sublevel set is empty, while for every finite $\eta \ge 0$ its sublevel set is $X^{\prime }$. Both are closed when $X^{\prime }$ is closed, so part ii proves that this indicator functional of a constraint set is lower semicontinuous.

Let $T_{\alpha }=A^{\ast }A+\alpha I$. For every $f\in X$,Thus $T_{\alpha }$ is a coercive operator. If it is invertible and $T_{\alpha }f=g$, the lower bound and the Cauchy-Schwarz inequality giveand therefore $\Vert T_{\alpha }^{-1}g\Vert \le {\alpha }^{-1}\Vert g\Vert$. Hence

Subtracting $T_{\alpha }{u}_n=z_n$ and $T_{\alpha }{u}_m=z_m$ and applying the coercive estimate from part i yieldsThus $(z_n)$ Cauchy implies $(u_n)$ Cauchy. Completeness of the Hilbert space $X$ gives $u_n\to u\in X$, and boundedness of $T_{\alpha }$ gives $z_n=T_{\alpha }{u}_n\to T_{\alpha }u$.

The lower bound in part i shows that $T_{\alpha }u=0$ implies $u=0$, so $T_{\alpha }$ is injective. Since $T_{\alpha }$ is self-adjoint,so its range is dense. Part ii shows that its range is also closed: a convergent sequence $z_n=T_{\alpha }{u}_n$ has Cauchy preimages, whose limit $u$ satisfies $T_{\alpha }u=\lim z_n$. Hence $\mathcal{R}(T_{\alpha })=X$. The operator is bijective, and the estimate in part i proves that its inverse is bounded. Therefore $A^{\ast }A+\alpha I$ is invertible for every $\alpha >0$.

The equation $(A^{\ast }A+{\alpha }_{\ast }I)u_{\ast }=A^{\ast }f$ is the Tikhonov normal equation. For $u=u_{\ast }+h$, expand the Tikhonov regularization functional:The normal equation makes the linear term zero. The final two terms are nonnegative and are strictly positive for $h\ne 0$ because ${\alpha }_{\ast }>0$. Thus $u_{\ast }$ is the unique global minimizer.

The objective separates by coordinates, so minimizeThe subgradient optimality condition isFor $z>0$ this gives $z=x_i-\lambda$, valid when $x_i>\lambda$; for $z<0$ it gives $z=x_i+\lambda$, valid when $x_i<-\lambda$. At $z=0$, the condition is $x_i\in [-\lambda ,\lambda ]$. Therefore the shrinkage operator is the soft-thresholding operator

The graph is continuous and piecewise linear: it is horizontal at zero on $[-\lambda ,\lambda ]$ and has slope one outside that interval, joining the axis at $(-\lambda ,0)$ and $(\lambda ,0)$.

 psi
  ^
  |                    /
  |                   /
--+---------==========--------> x
 /         -lambda  lambda
/