= Minimum size of a sufficient set for an IID source
{title2=$|S_n|\geq\frac{1-\epsilon}{2}2^{n(H-\delta)}$}
For finite-alphabet <independent and identically distributed random variables>, fixed $0\leq\epsilon<1$ and $\delta>0$, every <epsilon-sufficient set> $S_n$ of length-$n$ words satisfies $|S_n|\geq(1-\epsilon)2^{n(H-\delta)}/2$ for sufficiently large $n$. Intersect $S_n$ with a <typical set> whose excluded mass is at most $(1-\epsilon)/2$. The intersection retains that much probability and each word has probability at most $2^{-n(H-\delta)}$.
Back to article page
