Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 66 / 4 / ii / a / Solution

Let $Y_i=-{\log }_2{P}_Z(Z_i)$. Since the alphabet is finite, these independent and identically distributed random variables are finite almost surely, have mean $H(Z)$, and have finite variance $v$. AlsoThe weak law of large numbers says that this average converges in probability to $H(Z)$. ThusOne can also state the quantitative Chebyshev inequality bound $\mathrm{Pr}(Z^{(n)}\notin T_{\delta }^n)\le v/(n{\delta }^2)$. If $v=0$, every word of positive probability is in the typical set. This is the finite-alphabet asymptotic equipartition property.