Past exam of the mathematics course of the University of Cambridge / 2022 / iii / Paper 210 / 4

The pushforward measure of $\mu$ under $g$ isThe Lebesgue decomposition theorem says that for sigma-finite measures $P,Q$ there are unique measures $P_{\mathrm{ac}}$ and $P_{\mathrm{s}}$ such that

For a convex lower-semicontinuous function $f$ with $f(1)=0$, choose any measure $\lambda$ dominating probability measures $P,Q$, put $p=dP/d\lambda$ and $q=dQ/d\lambda$, and define the f-divergenceusing the lower-semicontinuous perspective value when $q=0$. This definition includes the singular part and is independent of $\lambda$.

To prove the data processing inequality for f-divergences, let $\mathcal{G}=\sigma (g)$ and use $\lambda =P+Q$. The densities of $g_{\ast }P$ and $g_{\ast }Q$, pulled back to $\mathcal{G}$, are ${\mathbb{E}}_{\lambda }(p\mid \mathcal{G})$ and ${\mathbb{E}}_{\lambda }(q\mid \mathcal{G})$. Since the perspectiveis jointly convex, conditional Jensen inequality givesIntegration proves

The chi-squared divergence iswhen $P\ll Q$, and is infinite otherwise.

Fix a probability measure $Q$. Let $J$ be uniform on $\{1,\dots ,M\}$ and, conditionally on $J=j$, draw $X$ from $P_j$. Compare this joint law with the reference law under which $J$ is uniform and independent of $X\sim Q$. Forthe target expectation is $M^{-1}{\sum }_j{P}_j(A_j)$, while its reference expectation isbecause the $A_j$ form a partition. Its reference variance is $M^{-1}(1-M^{-1})$. The Cauchy-Schwarz inequality applied to the likelihood ratio givesThe last divergence separates over $J$:Taking the infimum over all probability measures $Q$ proves the required inequality.