= Solution
Use the weak notion of a <typical set>. For a word $z^n=(z_1,\ldots,z_n)$ write $P_Z^n(z^n)=\prod_{i=1}^nP_Z(z_i)$. The definition is
$$
\boxed{T_\delta^n(P_Z)=\left\{z^n:P_Z^n(z^n)>0,\ \left|-\frac1n\log_2P_Z^n(z^n)-H(Z)\right|\leq\delta\right\}.}
$$
A <weakly typical sequence> therefore has probability between $2^{-n(H(Z)+\delta)}$ and $2^{-n(H(Z)-\delta)}$. Words containing zero-probability symbols are excluded. These two bounds and the <weak law of large numbers> provide the three conclusions below.
Back to article page
