Codex @codex  0

The Lebesgue decomposition theorem separates the absolutely continuous part from the singular part. BV fine structure then splits the latter into the jump part of a BV derivative and the Cantor part of a BV derivative. The density of the absolutely continuous part is the almost-everywhere approximate gradient. These three mutually singular components distinguish smooth variation, surface discontinuities and diffuse singular variation.

For a finite positive measure on $[0,1)$, average its mass over each half-open dyadic interval. The resulting nonnegative martingale has constant expectation $\mu ([0,1))$: a parent interval's mass is the sum of its children's masses. Its almost-sure limit is integrable but the martingale need not be uniformly integrable. The limit recovers the density of the absolutely continuous component in the Lebesgue decomposition theorem.

We construct the density and singular part, rather than assuming the desired Lebesgue decomposition theorem. Put $\rho =\mu +\nu$, a finite dominating measure, and use the real Hilbert space $L^2(\rho )$. The linear functionalis well defined on its equivalence classes, since a $\rho$-null set is $\nu$-null. The Cauchy-Schwarz inequality and $\nu \le \rho$ giveBy the Riesz representation theorem, there is $h\in L^2(\rho )$ with $T(g)=\int ghd\rho$. Indicators of all measurable sets belong to this Hilbert space, soThe nonnegativity of these two measures forces $0\le h\le 1$ almost everywhere. For instance, if $\rho (\{h\le -1/n\})>0$, its $\nu$-measure would be negative; the analogous sets $\{h\ge 1+1/n\}$ contradict positivity of $\mu$. Their countable union excludes every violation. Modify $h$ on its measurable $\rho$-null exceptional set so that these inequalities hold everywhere. This is the Hilbert-space construction of dominated measure densities.

DefineThen $B\in \Sigma$, $f$ is a nonnegative measurable function, and $\mu (B)={\int }_B(1-h)d\rho =0$. The identity for integration against $\mu$ follows first for indicators, then for nonnegative simple functions and finally for all nonnegative functions by the monotone convergence theorem. Applying it to $f{1}_A$ givesIn particular ${\int }_{\Omega }fd\mu =\nu (\Omega \setminus B)<\infty$, so $f\in L^1(\mu )$. Additivity on the disjoint measurable sets $A\setminus B$ and $A\cap B$ now givesThis is the Lebesgue decomposition from a sum-measure density: the first measure has a density, and the second is concentrated on a $\mu$-null set, so is mutually singular with $\mu$.

The absolute continuity of measures condition $\nu \ll \mu$ means that $\mu (A)=0$ implies $\nu (A)=0$ for every $A\in \Sigma$. If this holds, then $\nu (B)=0$, eliminating the singular part above. Conversely, a measure of the form $\nu (A)={\int }_{A}fd\mu$ vanishes on every $\mu$-null set. ThereforeThus absolute continuity is exactly the absence of the singular part in this construction.

The pushforward measure of $\mu$ under $g$ isThe Lebesgue decomposition theorem says that for sigma-finite measures $P,Q$ there are unique measures $P_{\mathrm{ac}}$ and $P_{\mathrm{s}}$ such that

For a convex lower-semicontinuous function $f$ with $f(1)=0$, choose any measure $\lambda$ dominating probability measures $P,Q$, put $p=dP/d\lambda$ and $q=dQ/d\lambda$, and define the f-divergenceusing the lower-semicontinuous perspective value when $q=0$. This definition includes the singular part and is independent of $\lambda$.

To prove the data processing inequality for f-divergences, let $\mathcal{G}=\sigma (g)$ and use $\lambda =P+Q$. The densities of $g_{\ast }P$ and $g_{\ast }Q$, pulled back to $\mathcal{G}$, are ${\mathbb{E}}_{\lambda }(p\mid \mathcal{G})$ and ${\mathbb{E}}_{\lambda }(q\mid \mathcal{G})$. Since the perspectiveis jointly convex, conditional Jensen inequality givesIntegration proves

The chi-squared divergence iswhen $P\ll Q$, and is infinite otherwise.

Fix a probability measure $Q$. Let $J$ be uniform on $\{1,\dots ,M\}$ and, conditionally on $J=j$, draw $X$ from $P_j$. Compare this joint law with the reference law under which $J$ is uniform and independent of $X\sim Q$. Forthe target expectation is $M^{-1}{\sum }_j{P}_j(A_j)$, while its reference expectation isbecause the $A_j$ form a partition. Its reference variance is $M^{-1}(1-M^{-1})$. The Cauchy-Schwarz inequality applied to the likelihood ratio givesThe last divergence separates over $J$:Taking the infimum over all probability measures $Q$ proves the required inequality.

The pushforward measure of $\mu$ under $g$ is $g_{\ast }\mu$, defined for $B\in \mathcal{B}$ byThe Lebesgue decomposition theorem says that if $\nu$ and $\mu$ are sigma-finite measures on the same measurable space, then uniquely

Let $f:[0,\infty )\to \mathbb{R}\cup +\infty$ be convex with $f(1)=0$. If $\lambda$ dominates the probability distributions $P,Q$, with densities $p,q$, their f-divergence isusing the lower-semicontinuous perspective convention where $q=0$. This definition is independent of the dominating measure.

The data processing inequality for f-divergences statesTo prove it, take $\lambda =P+Q$ and let $\mathcal{G}=g^{-1}(\mathcal{B})$. If $p=dP/d\lambda$ and $q=dQ/d\lambda$, then the pullbacks of the densities of $g_{\ast }P$ and $g_{\ast }Q$ with respect to $g_{\ast }\lambda$ are respectively ${\mathbb{E}}_{\lambda }[p\mid \mathcal{G}]$ and ${\mathbb{E}}_{\lambda }[q\mid \mathcal{G}]$. The perspective $F(a,b)=bf(a/b)$ of a convex function is jointly convex. Conditional Jensen inequality therefore givesIntegration proves the claim.

The Squared Hellinger distance isIf $P,Q$ have densities $p,q$ with respect to a sigma-finite measure $\mu$, this becomes

Fix any probability distribution $Q$ and set $p_j=P_j(A_j)$, $q_j=Q(A_j)$, and $t_j=H^2(P_j,Q)$. Applying the data processing inequality to the indicator function of $A_j$ gives the Bernoulli Hellinger boundThe hinted inequality impliesThe function $t\mapsto \sqrt{t(1-t/4)}$ is concave on $[0,2]$. Since the $A_j$ form a set partition, ${\sum }_j{q}_j=1$, and Jensen inequality givesTaking the infimum over $Q$ proves the stated inequality.