= Solution
Put $p=dP/d\mu$ and $q=dQ/d\mu$. Using natural <logarithms>, the <Kullback-Leibler divergence> is
$$
\boxed{K(P,Q)=\int_\Omega p\log\frac pq\,d\mu.}
$$
The integrand is zero where $p=0$, including where both densities vanish, and is $+\infty$ where $p>0=q$. Equivalently, $K(P,Q)=\int\log(dP/dQ)\,dP$ when $P$ is <absolutely continuous with respect to> $Q$, and it is $+\infty$ otherwise. This is a well-defined extended integral: on $p<q$, writing $t=p/q$ gives $-p\log(p/q)=-qt\log t\leq q/e$, so its negative part is integrable.
For <Pinsker's inequality>, an infinite <Kullback-Leibler divergence> makes the conclusion immediate. Assume it is finite. Set
$$
A=\{p\geq q\},\qquad a=P(A),\quad b=Q(A),\quad t=a-b.
$$
Since $\int(p-q)\,d\mu=0$, the positive and negative parts of $p-q$ have equal integrals. Thus
$$
\int|p-q|\,d\mu=2t,\qquad t\geq0.
$$
This $t$ is the <total variation distance> with the convention $\sup_E|P(E)-Q(E)|$.
We first prove the <binary partition bound for relative entropy>. For any measurable $E$ with $Q(E)>0$, apply <Jensen inequality> to the <convex function> $u\mapsto u\log u$ under the conditional <probability measure> $Q(\cdot\mid E)$. <Absolute continuity of measures> gives
$$
\int_E p\log\frac pq\,d\mu\geq P(E)\log\frac{P(E)}{Q(E)}.
$$
If $P(E)=0$, both sides are zero; if $Q(E)=0$, finiteness forces $P(E)=0$ as well. Apply this bound to $A$ and its complement to obtain
$$
K(P,Q)\geq d(a,b):=a\log\frac ab+(1-a)\log\frac{1-a}{1-b}.
$$
For $0<b<1$, regarding $d(a,b)$ as a <function> of $a$ gives
$$
d(b,b)=0,\qquad \partial_a d(b,b)=0,\qquad \partial_a^2d(a,b)=\frac1{a(1-a)}\geq4.
$$
Consequently $d(a,b)-2(a-b)^2$ is a <convex function> and has its minimum zero at $a=b$. By continuity this also holds at $a=0,1$. For $b=0,1$, the finite case has $a=b$ and the remaining cases have infinite divergence. This proves the <Bernoulli relative entropy lower bound> in every case. Combining the bounds yields
$$
K(P,Q)\geq2t^2=\frac12\left(\int|p-q|\,d\mu\right)^2,\qquad\boxed{\int|p-q|\,d\mu\leq\sqrt{2K(P,Q)}.}
$$
For the <Hellinger distance>, retain the unnormalized convention in which
$$
H^2(P,Q)=2-2\int\sqrt{pq}\,d\mu.
$$
The integral is the <Hellinger affinity>. In the finite-divergence case, $q/p>0$ for $P$-almost every point. Apply $-\log u\geq1-u$ with $u=\sqrt{q/p}$ to get
$$
\begin{aligned}
K(P,Q)&=\int_{\{p>0\}}-2\log\sqrt{q/p}\,p\,d\mu\\
&\geq2\int_{\{p>0\}}(1-\sqrt{q/p})p\,d\mu\\
&=2-2\int\sqrt{pq}\,d\mu=H^2(P,Q).
\end{aligned}
$$
The affinity is finite by <Cauchy-Schwarz inequality>. Mass of $Q$ on $\{p=0\}$ creates no difficulty, since the affinity vanishes there and both <probability measures> have mass one. The infinite-divergence case is again immediate. We obtain the <relative entropy bound for the unnormalized Hellinger distance>:
$$
\boxed{H(P,Q)\leq\sqrt{K(P,Q)}.}
$$
Back to article page
