Gibbs inequality
= Gibbs inequality
{c}
{wiki}
For <probability distribution>[probability distributions] $p$ and $q$ on the same finite set,
$$
D_{\mathrm{KL}}(p\Vert q)
=\sum_i p_i\log\frac{p_i}{q_i}\geq0,
$$
with equality exactly when $p=q$ on the support of $p$.