= Memoryless classical information source
{title2=$p(u^n)=\prod_{j=1}^np(u_j)$}
A memoryless classical source emits <independent and identically distributed random variables> from a fixed finite alphabet. Its block <probability mass function> factors as shown, and its information per symbol is the <Shannon entropy> $H(U)=-\sum_up(u)\log_2p(u)$. Zero-probability symbols never occur and contribute zero to the entropy. The <typical sequence theorem> describes the size and probability of the high-probability block set used for reliable compression.
Back to article page
