= Solution
The <independent and identically distributed> hypothesis factors the <joint probability mass function>:
$$
p_n(X_1,\ldots,X_n)=\prod_{i=1}^np(X_i),\qquad
\frac1n\log p_n(X_1,\ldots,X_n)=\frac1n\sum_{i=1}^n\log p(X_i).
$$
Symbols outside the <support of a probability distribution> $p$ never occur, almost surely. On its finite positive <probability support>, $\log p(X_i)$ is bounded, so the <strong law of large numbers> applies:
$$
\boxed{\frac1n\log p_n(X_1,\ldots,X_n)\longrightarrow
\sum_{x:p(x)>0}p(x)\log p(x)=-H(p)\quad\text{almost surely}.}
$$
This is the <asymptotic equipartition property> with the sign appropriate to a <logarithm> of a <probability mass>.
Back to article page
