Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 326 (Inverse Problems)

A problem is well posed in the sense of Hadamard when a solution exists for every admissible datum, is unique, and depends continuously on the datum. It is ill posed if any one of these three properties fails.

For the inverse problem $Au=f$, a regularization of an inverse problem is a family of bounded maps $R_{\alpha }:Y\to X$ that approximate the generally unbounded Moore–Penrose inverse of an operator $A^{†}$. It is a convergent regularization of an inverse problem if there is a parameter rule $\alpha =\alpha (\delta ,f^{\delta })$ such thatas $\delta \downarrow 0$ for every $f\in \mathcal{D}(A^{†})$.

Let $({\sigma }_i,x_i,y_i)$ be the singular system of a compact operator. Since $P_n{x}_i=x_i$ for $i\le n$ and $P_n{x}_i=0$ for $i>n$, the truncated operator $T_n=A{P}_n$ has the finite singular value decompositionConsequentlyso $\boxed{T_n^{†}=A^{†}{Q}_n}$.

For completeness, $T_n^{†}{T}_n=P_n$ on $(\ker A{)}^{\perp }$ and $T_n{T}_n^{†}=Q_n$. These are self-adjoint orthogonal projectors, and henceThus all four Penrose equations hold, which verifies the formula independently of the singular expansion.

Repeated substitution in Landweber iteration from $u_0=0$ givesOn the $i$th singular vector, $A^{\ast }A$ has eigenvalue ${\sigma }_i^2$ and $A^{\ast }{y}_i={\sigma }_i{x}_i$. The finite geometric series therefore givesThis is a spectral regularization method with filter

The regularization parameter is the stopping index $k$: increasing $k$ reduces the approximation bias but amplifies noise. Equivalently one may use the parameter $\alpha =1/k$, which tends to zero as $k\to \infty$.

It is sufficient thatand that the stopping rule obeyFor example, $k(\delta )=\lfloor {\delta }^{-1}\rfloor$ works.

For exact data $f\in \mathcal{D}(A^{†})$, each factor $(1-\tau {\sigma }_i^2{)}^k$ tends to zero. The Picard criterion makes $⟨f,y_i⟩/{\sigma }_i$ square summable, while $|1-(1-\tau {\sigma }_i^2{)}^k|$ is uniformly bounded. The dominated convergence theorem on the resulting series yields $u_k(f)\to A^{†}f$.

For noisy data with $\Vert f^{\delta }-f\Vert \le \delta$, the filter representation givesIndeed, when $\tau {\sigma }^2\le 1$, Bernoulli's inequality gives $1-(1-\tau {\sigma }^2{)}^k\le \min (k\tau {\sigma }^2,1)$; when $\tau {\sigma }^2>1$, the quotient is uniformly bounded because $\tau \Vert A{\Vert }^2<2$. Hence the triangle inequality giveswhich proves that early-stopped Landweber iteration is a convergent regularization of an inverse problem.

For $u\in L^1(\Omega )$, its total variation seminorm on a domain isThe corresponding bounded-variation space and its zero-mean subspace areIf $u_{\Omega }=|\Omega {|}^{-1}{\int }_{\Omega }udx$, the Poincaré inequality for total variation isIn particular, $\Vert u{\Vert }_{L^1}\le C_{\Omega }\mathrm{TV}(u)$ for $u\in B{V}_0(\Omega )$.

A $J$-minimizing solution is an exact solution $u_J^{†}$ satisfyingThe source condition in variational regularization says that some $p^{†}\in Y$ satisfieswhere $\partial J$ is the subdifferential.

Fix $\alpha >0$. The subgradient optimality condition forat $u_J^{†}$ isIf the source condition holds, choose $g=f+\alpha p^{†}$. Since $A{u}_J^{†}=f$, the displayed inclusion holds. Because the objective is a convex function, this condition is necessary and sufficient for global minimality.

Conversely, if the stated range condition holds for some $g$, optimality supplies $\xi \in \partial J(u_J^{†})$ withThus $\xi =A^{\ast }((g-f)/\alpha )$, which is precisely the source condition. This proves the equivalence.

For an absolutely one-homogeneous functional $J$, the subdifferential has the characterizationIf $c>0$, then $J(cu)=cJ(u)$, soThe second condition does not involve the base point. Therefore

Because $\lambda f\in \partial J(f)$ and $1-\lambda \alpha >0$ when $\alpha <1/\lambda$, part a givesSet $u=(1-\lambda \alpha )f$. ThenThis is the subgradient optimality condition forThe squared norm is strictly convex, so the objective has at most one minimizer. Henceis its unique minimizer.

A Bayesian inverse problem consists of a prior distribution ${\mu }_0$ on the unknown $u\in X$, a reference measure on the data space $Y$, and a jointly measurable likelihood $L(u;y)$ such that $L(u;\cdot )$ is a probability density function for ${\mu }_0$-almost every $u$. For observed data $y$, Bayes theorem defines the posterior distribution byprovided $0<Z(y)<\infty$.

The total variation distance isIf both measures have densities $p,q$ with respect to a common dominating measure, then $d_{\mathrm{TV}}(\mu ,\nu )=\frac{1}{2}\int |p-q|$.

The problem is a well-posed Bayesian inverse problem in total variation when a unique posterior ${\mu }^y$ exists for every $y\in Y$ and the posterior map is continuous:Thus the metric supplies the precise meaning of continuous dependence on the observed data in the Bayesian version of Hadamard well-posedness.

The following four assumptions are sufficient, with statements understood for ${\mu }_0$-almost every $u$ and every $y$:

The first two assumptions give $0<Z(y)<\infty$. The last two allow the dominated convergence theorem to prove $L^1({\mu }_0)$ continuity of the normalized posterior density, which is equivalent to continuity in total variation distance.

The noise has the product Laplace distribution densitywhich is normalized because ${\int }_{\mathbb{R}}{e}^{-2|t|}dt=1$. By translation invariance of Lebesgue measure, the conditional law of $A(u)+N$ has densityHence an associated likelihood function isThe map $(u,y)\mapsto y-A(u)$ is measurable because $A$ is measurable and vector subtraction is continuous. Composition with the continuous norm and exponential functions proves that $L$ is jointly measurable.

We may take the everywhere-defined representativeIt agrees with the likelihood from part a, hence certainly agrees ${\mu }_0\otimes {\lambda }_k$-almost everywhere. For every $u$, it is a strictly positive density in $y$ and is continuous in $y$. Moreoverso the constant function $h(u)=1$ is an integrable dominator for every probability measure ${\mu }_0$. All four sufficient assumptions from part 1d therefore hold, and the Bayesian inverse problem is well posed in total variation distance.

TakeThis is a finite measure, hence a sigma-finite measure. If $\nu (B)=0$, nonnegativity gives $\mu (B)={\mu }_0(B)=0$, so both $\mu$ and ${\mu }_0$ are absolutely continuous with respect to $\nu$.

Write $p=d\mu /d\rho$ and $q=d{\mu }_0/d\rho$, whose existence follows from the Radon-Nikodym theorem. Sincewe haveThus the defining integral is finite and the Hellinger distance is well defined.

For nonnegative $p,q$,Integrating and using the density formula for total variation distance gives

Every posterior given by Bayes theorem is absolutely continuous with respect to its prior ${\mu }_0$, so it belongs to $\mathrm{Prob}(X,{\mathcal{B}}_X,{\mu }_0)$. If $y_n\to y$, total-variation well-posedness givesPart c, now using the common dominating measure ${\mu }_0$, yieldsExistence and uniqueness are unchanged. The problem is therefore a well-posed Bayesian inverse problem in Hellinger distance.

A Gaussian measure $\mu$ on a real separable Banach space $X$ is a Borel probability measure such that ${\ell }_{\ast }\mu$ is a one-dimensional normal distribution for every continuous linear functional $\ell \in X^{\ast }$. Its mean $m\in X$ and covariance operator of a Gaussian measure $C:X^{\ast }\to X$ are characterized byandfor all ${\ell }_1,{\ell }_2\in X^{\ast }$.

The indicator functions satisfyTherefore ${\phi }_1,{\phi }_2\in L^2[0,1]$.

For every $h\in L^2[0,1]$, the continuous linear functional induced by the inner product givesThis is a normal random variable because it is a linear combination of independent normal random variables. Hence the law of $U$ is a Gaussian measure. Its mean is zero, and independence together with $\mathrm{Var}({\xi }_i)=1/2$ givesThus its covariance operator of a Gaussian measure is

The supports of ${\phi }_1$ and ${\phi }_2$ are disjoint, so they are orthogonal vectors. Substitution in the covariance formula givesandTherefore the associated eigenvalues are

Put $c_i={\int }_D{\phi }_i(x)d{\lambda }_n(x)$. The stated scalar random variable isAs a finite linear combination of independent normal random variables, it is normally distributed. Its mean is zero and its variance is

Finally, orthonormality of the ${\phi }_i$ givesSince $\mathbb{E}{\xi }_i^2=1/2$,