Past exam of the mathematics course of the University of Cambridge / 2014 / iii / Paper 7 / 4 / d / Solution

Put $u=F_N/{\gamma }_N$, where the Gaussian density is strictly positive. Extend $u\log u$ continuously at zero by $0\log 0=0$. Both densities have integral one, so the relative entropy can be written asThe bracket is nonnegative and vanishes only at $u=1$: its derivative for $u>0$ is $\log u$, with a unique minimum at one. HenceThe inequality holds also for infinite entropy. Its negative integrand part is integrable, since $u\log u\ge -1/e$ and ${\gamma }_N$ has integral one, so the extended-value integral is well defined. This is relative entropy in Kac's model; no differentiation is needed for nonnegativity.