Past exam of the mathematics course of the University of Cambridge / 2009 / ii / Paper 1 / 12H / i / Solution

For probability distributions $p,q$ with positive entries, Gibbs inequality iswith equality exactly when $p=q$. Indeed, $\log x\le x-1$, with equality only at $x=1$, givesThis also proves the equality condition. With zero entries the same Kullback-Leibler divergence statement holds using $0\log (0/q)=0$ and $p\log (p/0)=+\infty$ for $p>0$, or by restricting to the support and taking limits.