= Total variation–Hellinger–relative entropy inequality
{title2=$\operatorname{TV}(P,Q)\leq h(P,Q)\leq\sqrt{\operatorname{KL}(P,Q)}$}
For the <unnormalized Hellinger distance>, $\operatorname{TV}(P,Q)\leq h(P,Q)\leq\sqrt{\operatorname{KL}(P,Q)}$. Factor $|p-q|=|\sqrt p-\sqrt q|(\sqrt p+\sqrt q)$ and use the <Cauchy-Schwarz inequality> to bound the <total variation distance>. For the other bound, $\log t\leq t-1$ gives $p\log(p/q)\geq2(p-\sqrt{pq})$, whose integral is the squared <unnormalized Hellinger distance>. Infinite <Kullback-Leibler divergence> is allowed.
Back to article page
