Past exam of the mathematics course of the University of Cambridge / 2019 / iii / Paper 323 / 2 / i / b / Solution

Every member of the typical set has probability at least $2^{-n(H(X)+\epsilon )}$. Summing these probabilities givessoThe upper bound itself holds at every block length. For sufficiently large $n$, the high-probability property also gives the companion lower boundhence $|T_{\epsilon }^{(n)}|\ge (1-\delta )2^{n(H(X)-\epsilon )}$. Together these are the typical-set cardinality bounds underlying Shannon source coding theorem.