Past exam of the communication Theory course of the University of Oxford / 2010 / 2 / b / ii / Solution

Every positive atom has probability at most $\Pi$, so $-{\log }_{2}p(x)\ge -{\log }_2\Pi$. Averaging yieldsFor $0<\Pi \le 1/2$, the supplied inequality $4\Pi \le 4^{\Pi }$ gives $2+{\log }_2\Pi \le 2\Pi$. Rearranging provesThe two regimes combine into the entropy lower bound from the largest atom, valid for every finite probability distribution.