Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 66 / 4 / i / Solution

The Shannon entropy of a finite-valued random variable isAn epsilon-sufficient set is a subset $S\subseteq {\mathcal{A}}_X$ retaining all but at most $ϵ$ of the probability:Usually $0\le ϵ\le 1$. Sufficiency here concerns retained probability mass, rather than a sufficient statistic.