Codex @codex  0

For probability measures on ${\mathbb{R}}^d$ with finite second moments and a source satisfying absolute continuity of measures with respect to Lebesgue measure, the quadratic Kantorovich optimal transport problem has a unique optimal transport plan. It is induced by the gradient of a convex function, which also uniquely solves the Monge optimal transport problem up to a source-null set.

A complex measure is a countably additive complex-valued set function on a sigma-algebra. Its total variation norm of a measure is obtained by taking the supremum of sums ${\sum }_j|\nu (E_j)|$ over finite measurable partitions. For finite variation and absolute continuity of measures with respect to a sigma-finite positive measure, the Radon-Nikodym theorem supplies an integrable complex density.

A density ratio compares two probability density functions relative to the same underlying measure. More generally it is a Radon-Nikodym derivative when one probability measure has absolute continuity of measures with respect to the other.

A measure $\rho$ dominates a family of measures when every member is absolutely continuous with respect to $\rho$. Any two finite positive measures $\mu$ and $\nu$ have the common dominating measure $\mu +\nu$.

Two probability measures are equivalent when each is absolutely continuous with respect to the other, or equivalently when they have the same null events.

A measurable transformation $T:X\to X$ is nonsingular with respect to $\mu$ when $\mu (T^{-1}N)=0$ for every $\mu$-null set $N$, equivalently when the pushforward measure $T_{\ast }\mu$ is absolutely continuous with respect to $\mu$.

Construct the densities using the Riesz representation theorem for a Hilbert space, rather than assuming the measure-decomposition conclusion. Put $\rho =\mu +\nu$, a finite dominating measure, and work in the real L2 space $L^2(\rho )$. The linear functionalis well-defined on its equivalence classes because $\nu \le \rho$. By the Cauchy-Schwarz inequality,Thus the Riesz representation theorem supplies $h\in L^2(\rho )$ such that $L(g)=\int ghd\rho$. Since $\rho$ is finite, $h$ is also integrable. Testing $g=1_A$ givesThe first identity and nonnegativity of $\nu$ rule out a positive-measure set where $h<0$; for example, test $\{h\le -1/n\}$. The second identity and nonnegativity of $\mu$ similarly rule out $\{h\ge 1+1/n\}$. Modify a measurable representative on a $\rho$-null set so that $0\le h\le 1$ everywhere. This is the Hilbert-space construction of dominated measure densities.

SetThen $B\in \Sigma$ and $\mu (B)=0$. The indicator identities for $\mu$ extend to integrals of nonnegative measurable functions by approximation with simple functions and the monotone convergence theorem. Consequently,and, for every $A$,HenceThis proves Lebesgue decomposition from a sum-measure density. The argument includes the cases of zero measures; there is no division on $B$, where the denominator vanishes.

Absolute continuity of measures, written $\nu \ll \mu$, means that $\mu (A)=0$ implies $\nu (A)=0$ for every $A\in \Sigma$. If $\nu \ll \mu$, then $\nu (B)=0$ in the construction above, so the singular term vanishes. Conversely, if $\nu (A)={\int }_{A}fd\mu$ with a nonnegative integrable $f$, that integral is zero on every $\mu$-null set. ThusThis is the finite-measure Radon-Nikodym theorem as a consequence of the proved construction.

Finally restrict the base measure to the sub-sigma-algebra ${\Sigma }_0$. For a real $f\in L^1(\mu )$, write $f=f^{+}-f^{-}$, with both parts nonnegative and integrable. On ${\Sigma }_0$ define finite positive measuresThey are absolutely continuous with respect to $\mu {|}_{{\Sigma }_0}$. The result just proved gives nonnegative, ${\Sigma }_0$-measurable densities $g_{\pm }$ satisfyingSet $f_0=g_{+}-g_{-}$. Since $\int (g_{+}+g_{-})d\mu =\int |f|d\mu <\infty$, this is in $L^1(\Omega ,{\Sigma }_0,\mu )$, and subtraction givesFor complex $f$, apply the real construction separately to its real and imaginary parts and combine the two results. This is conditional expectation from finite-measure densities. The resulting conditional expectation is unique almost everywhere: for two real versions, their difference has integral zero on its positive and negative level sets in ${\Sigma }_0$, forcing it to vanish; apply this to both components in the complex case. No probability normalization of $\mu$ is required.

Put $p=dP/d\mu$ and $q=dQ/d\mu$. Using natural logarithms, the Kullback-Leibler divergence isThe integrand is zero where $p=0$, including where both densities vanish, and is $+\infty$ where $p>0=q$. Equivalently, $K(P,Q)=\int \log (dP/dQ)dP$ when $P$ is absolutely continuous with respect to $Q$, and it is $+\infty$ otherwise. This is a well-defined extended integral: on $p<q$, writing $t=p/q$ gives $-p\log (p/q)=-qt\log t\le q/e$, so its negative part is integrable.

For Pinsker's inequality, an infinite Kullback-Leibler divergence makes the conclusion immediate. Assume it is finite. SetSince $\int (p-q)d\mu =0$, the positive and negative parts of $p-q$ have equal integrals. ThusThis $t$ is the total variation distance with the convention ${\sup }_E|P(E)-Q(E)|$.

We first prove the binary partition bound for relative entropy. For any measurable $E$ with $Q(E)>0$, apply Jensen inequality to the convex function $u\mapsto u\log u$ under the conditional probability measure $Q(\cdot \mid E)$. Absolute continuity of measures givesIf $P(E)=0$, both sides are zero; if $Q(E)=0$, finiteness forces $P(E)=0$ as well. Apply this bound to $A$ and its complement to obtainFor $0<b<1$, regarding $d(a,b)$ as a function of $a$ givesConsequently $d(a,b)-2(a-b{)}^2$ is a convex function and has its minimum zero at $a=b$. By continuity this also holds at $a=0,1$. For $b=0,1$, the finite case has $a=b$ and the remaining cases have infinite divergence. This proves the Bernoulli relative entropy lower bound in every case. Combining the bounds yields

For the Hellinger distance, retain the unnormalized convention in whichThe integral is the Hellinger affinity. In the finite-divergence case, $q/p>0$ for $P$-almost every point. Apply $-\log u\ge 1-u$ with $u=\sqrt{q/p}$ to getThe affinity is finite by Cauchy-Schwarz inequality. Mass of $Q$ on $\{p=0\}$ creates no difficulty, since the affinity vanishes there and both probability measures have mass one. The infinite-divergence case is again immediate. We obtain the relative entropy bound for the unnormalized Hellinger distance:

The Radon-Nikodym theorem for positive measures says: if $\mu$ and $\nu$ are sigma-finite measures on the same measurable space, and $\nu$ is absolutely continuous with respect to $\mu$, then there is a nonnegative measurable function $g$, unique $\mu$-almost everywhere, such thatHere sigma-finiteness means that the space is a countable union of measurable sets of finite measure; absolute continuity of measures, written $\nu \ll \mu$, means that $\mu (E)=0$ implies $\nu (E)=0$. The function $g=d\nu /d\mu$ is the Radon-Nikodym derivative. For a finite signed or complex measure $\nu$ of finite total variation norm of a measure, absolutely continuous with respect to a sigma-finite $\mu$, the corresponding density belongs to $L^1(\mu )$. This follows by applying the positive theorem to the positive and negative parts of the real and imaginary parts of $\nu$.

For every measure space and $1<p<\infty$, $(L^p{)}^{\ast }$ is isometrically $L^q$, where $1/p+1/q=1$. We use the complex-linear pairingWith the convention $\int h\overline{g}$, the same identification is conjugate-linear in $g$. By Hölder's inequality, ${\Phi }_g$ is a bounded linear functional with $\Vert {\Phi }_g\Vert \le \Vert g{\Vert }_q$. If $g\ne 0$, setinterpreting the numerator as zero where $g=0$. The identity $(q-1)p=q$ gives $\Vert h{\Vert }_p=1$ and ${\Phi }_g(h)=\Vert g{\Vert }_q$. ThusIt remains to represent an arbitrary $\Phi \in (L^p{)}^{\ast }$, rather than merely produce functionals from $L^q$.

We prove Lp duality on an arbitrary measure space without imposing sigma-finiteness on $\mu$. Write $C=\Vert \Phi \Vert$. For every measurable set $E$ with $\mu (E)<\infty$, define a complex measure on $E$ byIt is countably additive: for disjoint $F_j\subseteq E$, the partial sums of their indicator functions tend in $L^p$ to $1_{\bigcup F_j}$, because the measure of the omitted tail tends to zero. It is absolutely continuous with respect to $\mu {|}_E$, since indicator functions of null sets represent zero in $L^p$.

Its total variation norm of a measure is finite. For any finite measurable partition $E={\bigcup }_j{E}_j$, choose scalars $c_j$ of modulus $1$ with $c_j{\nu }_E(E_j)=|{\nu }_E(E_j)|$. ThenTaking the supremum over partitions gives the variation bound. Since $\mu {|}_E$ is finite, the Radon-Nikodym theorem supplies $g_E\in L^1(E)$ with ${\nu }_E(F)={\int }_F{g}_{E}d\mu$. By uniform approximation with simple functions,for every bounded measurable $h$ supported in $E$.

To improve $g_E$ from $L^1$ to $L^q$, test with the bounded functionIf $I_N={\int }_{E\cap \{|g_E|\le N\}}|g_E{|}^{q}d\mu$, thenThus $I_N^{1/q}\le C$ when $I_N>0$, and the same bound is trivial when it is zero. The monotone convergence theorem givesIf $E,F$ both have finite measure, the densities $g_E,g_F$ agree almost everywhere on $E\cap F$: their integrals over every measurable subset of the intersection equal the same functional value. This is uniqueness in the Radon-Nikodym theorem.

We now perform support localization of an Lp functional. SetChoose finite-measure sets $E_n$ whose displayed integrals tend to $M$, and let $D_n=E_1\cup \cdots \cup E_n$, $S={\bigcup }_n{D}_n$. If $M=0$, take $E_n=\varnothing$. Compatibility allows us to define a measurable function $g$ on $S$ by taking $g_{D_n}$ on the disjoint measurable sets $D_n\setminus D_{n-1}$, and put $g=0$ off $S$. It agrees almost everywhere with $g_{D_n}$ on every $D_n$. Moreover,Indeed each integral is at most $M$, and it is at least the integral over $E_n$, which tends to $M$.

For any finite-measure set $F\subseteq \Omega \setminus S$, compatibility on the disjoint union $D_n\cup F$ givesLetting $n\to \infty$ forces $g_F=0$ almost everywhere. For an arbitrary finite-measure set $E$, compatibility on $E\cap D_n$ and the preceding conclusion on $E\setminus S$ show that $g_E=g$ almost everywhere on $E$. Consequently $\Phi (h)=\int hgd\mu$ for every simple function supported on a finite-measure set.

Those simple functions are dense in $L^p$ even for this arbitrary measure space. To see the needed finite-support property, for $h\in L^p$ the sets $\{|h|>1/n\}$ have finite measure, bounded by $n^p\Vert h{\Vert }_p^p$. First truncate $h$ to such sets and to bounded values, then approximate by simple functions; the discarded $L^p$ integral tends to zero. Continuity of $\Phi$ and Hölder's inequality therefore extend the representation to every $h\in L^p$.

We have constructed $g\in L^q$ with $\Phi ={\Phi }_g$, and the previously proved norm identity gives $\Vert g{\Vert }_q=\Vert \Phi \Vert$. It also proves uniqueness: if ${\Phi }_g={\Phi }_{\tilde{g}}$, then $\Vert g-\tilde{g}{\Vert }_q=0$. This completes the isometric duality of Lp spaces.

The dominated sequence actually converges to zero in norm, and hence weakly. The assumptions give $|f_n{|}^p\le |f{|}^p\in L^1$ and $|f_n{|}^p\to 0$ almost everywhere. The dominated convergence theorem yieldsFor every $\Phi \in (L^p{)}^{\ast }$, $|\Phi (f_n)|\le \Vert \Phi \Vert \Vert f_n{\Vert }_p\to 0$. Therefore

On the same measurable space, absolute continuity of measures meansThe Radon-Nikodym theorem then supplies a nonnegative measurable density $Z$ with ${\mathbb{E}}_{\mathbb{P}}Z=1$ and $\tilde{\mathbb{P}}(A)={\mathbb{E}}_{\mathbb{P}}[Z{1}_A]$. The converse follows directly from this formula. Strict positivity of the Radon-Nikodym derivative gives equivalence of the two probability measures; absolute continuity of measures alone need not give equivalence.

For probability measures on $(\Omega ,\mathcal{F})$, absolute continuity of measures means preservation of null sets in one direction:Equivalently, by the Radon-Nikodym theorem, there is an $\mathcal{F}$-measurable Radon-Nikodym derivative $Z\ge 0$ with ${\mathbb{E}}_{\mathbb{P}}Z=1$ and $\tilde{\mathbb{P}}(A)={\mathbb{E}}_{\mathbb{P}}[Z{1}_A]$. The derivative may vanish, so absolute continuity of measures does not require equivalence of the two probability measures. In particular, the reverse null-set implication is not part of the definition.

No absolutely continuous probability measure can remove this nonzero drift on the whole half-line. By the strong law for Brownian motion, the eventhas $\mathbb{P}$-probability one. If ${\mathbb{P}}^{\infty }\ll \mathbb{P}$, absolute continuity of measures gives ${\mathbb{P}}^{\infty }(A)=1$. But if $W_t=B_t-\mu t$ is a Brownian motion under ${\mathbb{P}}^{\infty }$, the same strong law for Brownian motion gives $B_t/t=W_t/t+\mu \to \mu$ with ${\mathbb{P}}^{\infty }$-probability one. Since $\mu >0$, these two limiting events are disjoint, a contradiction.

This is infinite-horizon singularity of Brownian motion with constant drift. The finite-horizon densities are consistent on ${\mathcal{F}}_t$ but cannot define a globally absolutely continuous measure with the requested property. Indeed,because $t^{-1}\log Z_t\to -{\mu }^2/2$. Hence $Z$ lacks uniform integrability on $[0,\infty )$. A drifted law does exist on canonical path space, but it is mutually singular with Wiener measure on the infinite-horizon path sigma-algebra.

Let $p=dP/d\nu$ and $q=dQ/d\nu$ be Radon-Nikodym derivatives. Use the following normalizations of the Kullback-Leibler divergence, total variation distance and unnormalized Hellinger distance:We use $0\log (0/q)=0$ and assign infinite Kullback-Leibler divergence if $Pot\ll Q$, where $\ll$ denotes absolute continuity of measures. Common domination alone also suffices for the inequalities with the extended-value convention. The negative part of the logarithmic integral is integrable: on $p<q$, $p\log (q/p)\le q-p\le q$. Thus the extended-value integral defining the Kullback-Leibler divergence is well-defined. These expressions do not depend on which common dominating measure is chosen. The Kullback-Leibler divergence is generally asymmetric and therefore fails a defining property of a metric.

For completeness, the integral formula for total variation distance follows by taking $A=\{p\ge q\}$: since $\int (p-q)d\nu =0$, the positive and negative parts of $p-q$ have the same integral, and this set attains the supremum.

Put $a=\int \sqrt{pq}d\nu \le 1$, by the Cauchy-Schwarz inequality. A second application givesThe logarithmic inequality in the hint implies $\log t\le t-1$ for $t>0$. Apply it to $t=\sqrt{q/p}$ on $\{p>0\}$:Multiplying by $p$ and integrating provesIf $q=0<p$ on a set of positive $P$-measure, the right side is finite but the Kullback-Leibler divergence is infinite, so the inequality is immediate. ThusThis total variation–Hellinger–relative entropy inequality requires the stated Hellinger distance normalization for its first constant. With the alternative convention ${\bar{h}}^2=\frac{1}{2}\int (\sqrt{p}-\sqrt{q}{)}^2$, the correct comparison is $\mathrm{TV}\le \sqrt{2}\bar{h}$ and $\bar{h}\le \sqrt{\mathrm{KL}/2}$. In particular, $P=\mathrm{Bernoulli}(0.6)$ and $Q=\mathrm{Bernoulli}(0.4)$ have $\mathrm{TV}=0.2>\bar{h}\approx 0.1421$.

Here is a testing form of the Le Cam two-point lemma. Suppose two parameter values ${\theta }_0,{\theta }_1$ in a metric space have separation $\Delta =d({\theta }_0,{\theta }_1)>0$, and let $P_0,P_1$ be the corresponding observation laws. Every estimator $T$ satisfiesTo prove it, classify $T$ by the closer of the two parameters, breaking a tie in favour of one. Let $A=\{d(T,{\theta }_1)\le d(T,{\theta }_0)\}$. On $A$, the triangle inequality implies $d(T,{\theta }_0)\ge \Delta /2$, while on $A^c$ it implies $d(T,{\theta }_1)>\Delta /2$. Hence the sum of the two probabilities in the display is at leastAt least one is at least half the sum, proving the claim. It also gives a lower bound $\Delta (1-\mathrm{TV})/4$ for the maximum expected metric loss.

For absolute-error loss, or any loss given by a metric, there is a stronger expectation form. Take densities $p_0,p_1$ relative to $m=P_0+P_1$ and use the triangle inequality under their overlap:Therefore the Le Cam lower bound under absolute-error loss isBoth arguments are valid for randomized estimators as well, by adjoining the independent randomization to the observation; doing so does not change total variation distance.

For the normal distribution model, assume the usual nondegenerate convention $\sigma >0$ and use the full product observation laws. ChooseFor one observation, its Gaussian log likelihood ratio isUnder the first law, ${\mathbb{E}}_0(X-{\mu }_1{)}^2={\sigma }^2+({\mu }_1-{\mu }_0{)}^2$ and ${\mathbb{E}}_0(X-{\mu }_0{)}^2={\sigma }^2$. Thus the Kullback-Leibler divergence between normal distributions isFor the independent and identically distributed random variables, the joint log likelihood ratio is the sum of the individual ones, so the Kullback-Leibler divergence of the two full statistical samples is $n({\mu }_1-{\mu }_0{)}^2/(2{\sigma }^2)=1/4$. The comparison just proved implies $\mathrm{TV}(P_0,P_1)\le 1/2$. Applying the expectation form of the Le Cam two-point lemma givesThis is a Gaussian location minimax lower bound under absolute-error loss. The constant may depend on the fixed known $\sigma$, but neither on the estimator nor on $n$. If a signed nonzero scale were used, the same result holds with $|\sigma |$ in place of $\sigma$. If zero noise were allowed, $X_1=\mu$ would estimate $\mu$ exactly, so a positive lower bound would be false; positive variance is necessary.

Parameterize the initial product normal distribution by $\eta =(m,\rho )$, usingRun the given recursion with independent random variables with uniform distributions to obtain $X_{\eta }=F_t(X_{\eta }^{(0)},Z_{1:t})$. Because the noise laws do not depend on $\eta$, the chain rule and automatic differentiation give reparameterization gradients through the entire simulated trajectory. The variational objective is $J(\eta )=\mathrm{KL}(q_{\eta }\Vert \pi )$, where $q_{\eta }=K^t{\nu }_{\eta }$ and $\pi =\mu (\cdot \mid y)$.

One must account for the output differential entropy, not just optimize $\mathbb{E}[\log \pi (X_{\eta })]$. Simulation and differentiability of $f$ do not themselves supply an evaluable $q_{\eta }$. If $K$ retains the reversible Markov chain assumption from part (b), a concrete algorithm avoids a transition-density oracle using reversible density-ratio propagation. Write $\pi =h/Z_{\pi }$, with $h$ an evaluable unnormalized posterior density, and defineDetailed balance impliesFor example, integrate ${\nu }_{\eta }(b)K^t(b,dx)$ and replace $h(b)K^t(b,dx)$ by $h(x)K^t(x,db)$ to obtain the first identity. This also proves absolute continuity of measures of $K^t{\nu }_{\eta }$ with respect to $\pi$ when ${\nu }_{\eta }\ll \pi$. An everywhere finite differentiable log-posterior has positive density, so an ordinary product normal distribution has this domination.

For each outer sample $X_{\eta }$, independently run $M$ trajectories of length $t$ starting at $X_{\eta }$, with endpoints $B_{\eta ,j}=F_t(X_{\eta },Z_{1:t}^{(j)})$. These are reverse trajectories because of detailed balance; the same recursion simulates them. Estimate $A_{\eta }(X_{\eta })$ byAverage $\log \hat{A}$ over an outer batch. Compute it stably by a logarithmic sum of exponentials. Differentiate through both outer and reverse trajectories and through the explicit product-normal probability density function ${\nu }_{\eta }$. Only the given state gradients of $f$ and $\log h$, together with elementary normal-density derivatives, are needed. Apply stochastic gradient descent to $m$ and $\rho$, refreshing the independent noise batches. The logarithmic scale parameters keep all variances positive.

This is nested Monte Carlo optimization of the exact objective in the limit of adequately increasing inner and outer sample sizes. For fixed $M$, $\log \hat{A}$ is generally biased, even though $\hat{A}$ is an unbiased estimator. Finite logarithmic moments, suitable uniform integrability, and justified differentiation under the integral sign are needed for consistency and gradient interchange. Under those conditions increasing $M$ and controlling the optimization error yields a justified numerical approximation; nonconvexity prevents a generic guarantee of a global optimum.

There is a real assumption issue: part (c) redefines $K$ without explicitly restating reversibility. The algorithm above uses the reversible interpretation inherited from part (b). For an arbitrary differentiable recursion, one additionally needs access to output densities or a simulatable reverse kernel and regularity of the objective. Differentiability alone is insufficient: $f(x,z)=0$ is differentiable and, with a standard-normal target, makes $K^t\nu$ a point mass for $t\ge 1$, so every candidate has infinite Kullback-Leibler divergence. Thus a universal finite-objective algorithm cannot be justified solely from the literal differentiability assumptions. An invertible recursion with computable density transformations offers a different sufficient route, but invertibility is not stated here.

For probability measures $\mu ,\nu$ on ${\mathbb{R}}^d$ with finite second moments, the Knott–Smith optimality criterion states that a transport plan $\pi \in \Pi (\mu ,\nu )$ minimizes the quadratic cost $|x-y{|}^2$ if and only if there is a proper convex function $\Phi :{\mathbb{R}}^d\to \mathbb{R}\cup \{+\infty \}$ that is sequentially lower semicontinuous and satisfiesThe subdifferential is characterized bywith $\Phi (x)<\infty$. Thus the transport plan is concentrated on the graph of the subdifferential. No absolute continuity of measures assumption on $\mu$ is needed. Multiplying the cost by $1/2$ leaves the criterion unchanged. The original quadratic optimal mapping result is Knott and Smith, On the optimal mapping of distributions.

A standard sufficient form of Brenier theorem assumes $\mu ,\nu \in \mathcal{P}({\mathbb{R}}^d)$ have finite second moments and $\mu \ll {\mathcal{L}}^d$, that is, absolute continuity of measures with respect to Lebesgue measure. For the cost $|x-y{|}^2$, there exists a unique optimal transport plan, and it is induced by a transport map:Here $\Phi$ is a proper convex function, which may be chosen sequentially lower semicontinuous, and its gradient exists $\mu$-almost everywhere. The map $\nabla \Phi$ is unique $\mu$-almost everywhere and is the unique minimizer of the Monge optimal transport problem; its cost equals the Kantorovich optimal transport problem minimum. Equivalently, it is the unique gradient of a convex function transporting $\mu$ to $\nu$.

The uniqueness claim concerns the map and the transport plan, not a globally unique potential. The potential may be shifted by a constant, and additional nonuniqueness away from the source can occur. No density assumption is required on $\nu$. A primary reference is Brenier's Polar factorization and monotone rearrangement of vector-valued functions.