= Solution
Let $Y_i=-\log_2P_Z(Z_i)$. Since the alphabet is finite, these <independent and identically distributed random variables> are finite almost surely, have mean $H(Z)$, and have finite variance $v$. Also
$$
-\frac1n\log_2P_Z^n(Z^{(n)})=\frac1n\sum_{i=1}^nY_i.
$$
The <weak law of large numbers> says that this average converges in probability to $H(Z)$. Thus
$$
\boxed{\lim_{n\to\infty}\Pr\bigl(Z^{(n)}\in T_\delta^n(P_Z)\bigr)=1\quad(\delta>0).}
$$
One can also state the quantitative <Chebyshev inequality> bound $\Pr(Z^{(n)}\notin T_\delta^n)\leq v/(n\delta^2)$. If $v=0$, every word of positive probability is in the <typical set>. This is the finite-alphabet <asymptotic equipartition property>.
Back to article page
