Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 66 / 4 / ii / Solution

Use the weak notion of a typical set. For a word $z^n=(z_1,\dots ,z_n)$ write $P_Z^n(z^n)={\prod }_{i=1}^n{P}_Z(z_i)$. The definition isA weakly typical sequence therefore has probability between $2^{-n(H(Z)+\delta )}$ and $2^{-n(H(Z)-\delta )}$. Words containing zero-probability symbols are excluded. These two bounds and the weak law of large numbers provide the three conclusions below.