Past exam of the mathematics course of the University of Cambridge / 2021 / iii / Paper 326 / 3

A Bayesian inverse problem consists of a prior distribution ${\mu }_0$ on the unknown $u\in X$, a reference measure on the data space $Y$, and a jointly measurable likelihood $L(u;y)$ such that $L(u;\cdot )$ is a probability density function for ${\mu }_0$-almost every $u$. For observed data $y$, Bayes theorem defines the posterior distribution byprovided $0<Z(y)<\infty$.

The total variation distance isIf both measures have densities $p,q$ with respect to a common dominating measure, then $d_{\mathrm{TV}}(\mu ,\nu )=\frac{1}{2}\int |p-q|$.

The problem is a well-posed Bayesian inverse problem in total variation when a unique posterior ${\mu }^y$ exists for every $y\in Y$ and the posterior map is continuous:Thus the metric supplies the precise meaning of continuous dependence on the observed data in the Bayesian version of Hadamard well-posedness.

The following four assumptions are sufficient, with statements understood for ${\mu }_0$-almost every $u$ and every $y$:

The first two assumptions give $0<Z(y)<\infty$. The last two allow the dominated convergence theorem to prove $L^1({\mu }_0)$ continuity of the normalized posterior density, which is equivalent to continuity in total variation distance.

The noise has the product Laplace distribution densitywhich is normalized because ${\int }_{\mathbb{R}}{e}^{-2|t|}dt=1$. By translation invariance of Lebesgue measure, the conditional law of $A(u)+N$ has densityHence an associated likelihood function isThe map $(u,y)\mapsto y-A(u)$ is measurable because $A$ is measurable and vector subtraction is continuous. Composition with the continuous norm and exponential functions proves that $L$ is jointly measurable.

We may take the everywhere-defined representativeIt agrees with the likelihood from part a, hence certainly agrees ${\mu }_0\otimes {\lambda }_k$-almost everywhere. For every $u$, it is a strictly positive density in $y$ and is continuous in $y$. Moreoverso the constant function $h(u)=1$ is an integrable dominator for every probability measure ${\mu }_0$. All four sufficient assumptions from part 1d therefore hold, and the Bayesian inverse problem is well posed in total variation distance.

TakeThis is a finite measure, hence a sigma-finite measure. If $\nu (B)=0$, nonnegativity gives $\mu (B)={\mu }_0(B)=0$, so both $\mu$ and ${\mu }_0$ are absolutely continuous with respect to $\nu$.

Write $p=d\mu /d\rho$ and $q=d{\mu }_0/d\rho$, whose existence follows from the Radon-Nikodym theorem. Sincewe haveThus the defining integral is finite and the Hellinger distance is well defined.

For nonnegative $p,q$,Integrating and using the density formula for total variation distance gives

Every posterior given by Bayes theorem is absolutely continuous with respect to its prior ${\mu }_0$, so it belongs to $\mathrm{Prob}(X,{\mathcal{B}}_X,{\mu }_0)$. If $y_n\to y$, total-variation well-posedness givesPart c, now using the common dominating measure ${\mu }_0$, yieldsExistence and uniqueness are unchanged. The problem is therefore a well-posed Bayesian inverse problem in Hellinger distance.