Past exam of the mathematics course of the University of Cambridge / 2018 / iii / Paper 326 / 1

Use the Hilbert space interpretation of a linear inverse problem, with a bounded linear operator $K:\mathcal{U}\to \mathcal{V}$. A least-squares solution minimizes $\Vert Ku-f{\Vert }_{\mathcal{V}}$ over $u\in \mathcal{U}$. A minimum-norm least-squares solution additionally has the smallest $\mathcal{U}$-norm among all least-squares solutions. When the equation is consistent this is the smallest-norm exact solution.

The least-squares solutions form a nonempty closed affine subspace precisely whenThe closest point theorem in a Hilbert space then gives a unique nearest point to zero, hence a unique minimum-norm least-squares solution. Equivalently, it is the unique least-squares solution in $\mathcal{N}(K{)}^{\perp }$. If $\mathcal{R}(K)$ is closed, this exists for every datum; a nonclosed operator range can leave some data without any least-squares solution.

For example, on the l2 sequence space, take $(Ku{)}_n=u_n/n$ and $f_n=1/n$. The equation would require $u_n=1$ for every $n$, which is not square summable. However, setting the first $N$ coordinates of $u$ to one and the rest to zero givesThus the least-squares infimum is zero but is not attained, and no minimum-norm least-squares solution exists for these data.

The map $K^{\ast }K$ is a bounded linear operator, so the inverse image of the closed singleton $\{K^{\ast }f\}$ is closed. That inverse image is also a convex set: if $K^{\ast }Ku=K^{\ast }Kv=K^{\ast }f$, every convex combination has the same image.

The identity $\mathcal{N}(K^{\ast })=\mathcal{R}(K{)}^{\perp }$ givesConsequently such a $u$ exists exactly when $f$ is the sum of an element of $\mathcal{R}(K)$ and one of its orthogonal complement. ThusThese are precisely the least-squares solutions. Indeed, if $r=Ku-f$ and $K^{\ast }r=0$, then for every $h$ the Pythagorean identity givesConversely, differentiating the squared residual along any real direction $h$ forces $\mathrm{Re}⟨K^{\ast }r,h⟩=0$, hence the normal equation for a linear inverse problem. For any $u_{\ast }\in \mathbb{L}$, the difference of two solutions of the normal equation for a linear inverse problem lies in $\mathcal{N}(K)$, since $⟨K^{\ast }Kh,h⟩=\Vert Kh{\Vert }^2$. Thereforea closed affine subspace; the empty case is closed and convex as well.

The Moore–Penrose inverse of an operator is the linear mapthat assigns each admissible datum its minimum-norm least-squares solution. If $f=y+z$ with $y\in \mathcal{R}(K)$ and $z\in \mathcal{R}(K{)}^{\perp }$, then $K^{†}f$ is the unique $u\in \mathcal{N}(K{)}^{\perp }$ satisfying $Ku=y$. In particular it vanishes on $\mathcal{R}(K{)}^{\perp }$, andThe continuity criterion, with the inherited data norm on its domain, isFor a closed operator range, the restriction of $K$ from $\mathcal{N}(K{)}^{\perp }$ to $\mathcal{R}(K)$ is a bounded bijection between Banach spaces, so its inverse is bounded by the bounded inverse theorem. Conversely, if $K^{†}$ is bounded and $K{u}_n\to y$, replace each $u_n$ by $P_{\mathcal{N}(K{)}^{\perp }}{u}_n=K^{†}K{u}_n$. This is a Cauchy sequence, hence converges to $u$, and $Ku=y$. Thus every limit of range elements remains in the operator range.

Write $u^{†}=K^{†}f$. Every least-squares solution is $u^{†}+w$ for some $w\in \mathcal{N}(K)$, while $u^{†}\in \mathcal{N}(K{)}^{\perp }$. Decompose $u_0=P_{\mathcal{N}(K)}{u}_0+P_{\mathcal{N}(K{)}^{\perp }}{u}_0$ using orthogonal projections. Then the Pythagorean identity givesThe first term is independent of $w$, and the second vanishes at exactly one point. Hence the unique nearest least-squares solution isGeometrically this is the orthogonal projection of $u_0$ onto the closed affine subspace of least-squares solutions.

A finite-valued global minimizer of the functional $E$ is an element $\bar{u}\in \mathrm{dom}E$ satisfyingHere $\mathrm{dom}E$ is its effective domain. The finite-value convention avoids treating an infeasible problem as solved.

The functional is a proper extended-real function if its effective domain is nonempty; the specified codomain already excludes $-\infty$. It has coercivity if $\Vert u_n\Vert \to \infty$ always implies $E(u_n)\to +\infty$, equivalently every finite sublevel set is bounded. It is $\tau$-sequentially lower semicontinuous ifThe topology in this definition matters: norm sequential lower semicontinuity and weak sequential lower semicontinuity need not coincide for a nonconvex functional.

On $\mathcal{U}=\mathbb{R}$ with its usual topology, the following examples isolate the three failures.

A nonproper functional is $E(x)=+\infty$ for every $x$. Its effective domain is empty, so it has no finite-valued global minimizer. It nevertheless has coercivity and is sequentially lower semicontinuous. If a minimizer is instead defined only by $E(x)=\inf E$ without requiring finiteness, every point formally minimizes this function; under that convention the requested nonproper counterexample is impossible with the given codomain.

A noncoercive functional is $E(x)=e^x$. It is a proper extended-real function and is continuous, but its infimum zero is approached as $x\to -\infty$ and is never attained. Hence it has no global minimizer.

A failure of sequential lower semicontinuity isIt is a proper extended-real function with coercivity, but $E(1/n)\to 0<E(0)$. Its infimum is zero, while every function value is positive. Thus it has no global minimizer.

The epigraph isAssume $E$ is $\tau$-sequentially lower semicontinuous. If $(u_n,t_n)$ lies in the epigraph and converges to $(u,t)$ in the product topology, thenThe limit remains in the epigraph, so it is a sequentially closed set.

Conversely, suppose the epigraph is a sequentially closed set and sequential lower semicontinuity fails along $u_n\stackrel{\tau }{\to }u$. Choose a finite real number $a$ with $\mathrm{liminf}E(u_n)<a<E(u)$. There is a subsequence along which $E(u_{n_j})\le a$. Thus $(u_{n_j},a)$ lies in the epigraph and converges to $(u,a)$, which does not lie there. This contradiction provesThis is a sequential statement; identifying it with ordinary closedness requires an appropriate assumption on the topology.

The direct method in the calculus of variations gives the following existence theorem. Suppose bounded sequences in the Banach space $\mathcal{U}$ have $\tau$-convergent subsequences, and $E$ is a proper extended-real function with coercivity and is $\tau$-sequentially lower semicontinuous. Then $E$ has a finite-valued global minimizer.

To prove this, properness makes $m=\inf E<+\infty$. First rule out $m=-\infty$: a sequence with $E(u_n)\to -\infty$ eventually lies in a fixed sublevel set, which is bounded by coercivity. A $\tau$-convergent subsequence would then have a limit $u$ with $E(u)\le -\infty$, contradicting the codomain. Hence $m$ is finite.

Choose a minimizing sequence with $E(u_n)\to m$. It eventually belongs to the bounded sublevel set $\{E\le m+1\}$. Extract $u_{n_j}\stackrel{\tau }{\to }\bar{u}$. By sequential lower semicontinuity,Thus $E(\bar{u})=m$. In particular, a reflexive Banach space with the weak topology supplies the required subsequence property by weak sequential compactness of bounded sequences in a reflexive Banach space. A strictly convex function has at most one minimizer; this is an additional property, not part of the existence theorem.

Take the weak topology on the reflexive Banach space $\mathcal{U}$. A bounded linear operator is weak-to-weak continuous: $u_n\rightharpoonup u$ implies $⟨K{u}_n,v⟩\to ⟨Ku,v⟩$ for every $v\in \mathcal{V}$, by its adjoint operator. The weak lower semicontinuity of the Hilbert norm therefore makes $D(u)=\Vert Ku-f\Vert$ weakly sequentially lower semicontinuous.

This functional is finite everywhere and nonnegative. If it has coercivity, the direct method in the calculus of variations supplies a global minimizer $\bar{u}$. Squaring the residual does not change its global minimizers. Here the adjoint operator has codomain ${\mathcal{U}}^{\ast }$, since the source is a Banach space. Differentiating the squared residual along every real direction $h\in \mathcal{U}$ gives $\mathrm{Re}⟨K^{\ast }(K\bar{u}-f),h⟩=0$ in the dual pairing, hence the normal equation for a linear inverse problem $K^{\ast }(K\bar{u}-f)=0$. HenceThe converse fails. For $K:{\mathbb{R}}^2\to \mathbb{R}$, $K(x,y)=x$, and $f=1$, the datum is in the operator range, but $D(1,n)=0$ while $\Vert (1,n)\Vert \to \infty$. Thus admissible data do not imply coercivity of the residual; an unpenalized null space already prevents it.