Past exam of the information theory course of the University of Oxford / 2023 / 1 / c / Solution

For the entropy summand $f(x)=-x\log x$, differentiation on $(0,1)$ givesSince $f(x)\to 0$ as $x\downarrow 0$, its continuous extension to $[0,1]$ is a concave function.

Fix $0<v<1$ and define the increment $D_v(t)=f(t+v)-f(t)$ on $[0,1-v]$. In its interior,Thus the increment is decreasing, and its endpoint values are $D_v(0)=f(v)$ and $D_v(1-v)=-f(1-v)$. ThereforeThe cases $v=0,1$ follow directly from the endpoint values.

To obtain the sharper half-interval bound, put $G(v)=f(v)-f(1-v)$. It has $G(0)=G(1/2)=0$, whileHence $G$ is a concave function on $[0,1/2]$ and lies above the chord joining its zero endpoints. Thus $f(v)\ge f(1-v)$ there, proving