= Shannon's block-entropy approximations
{c}
In <A Mathematical Theory of Communication by Claude Shannon>, <Claude Shannon> compares block uncertainty per symbol, denoted $G_N$, with conditional uncertainty given the preceding symbols, denoted $F_N$. Theorems 5 and 6 connect both decreasing approximations to the source's limiting uncertainty. He uses symbol statistics at increasing block lengths to expose structure missed by single-letter counts.<A Mathematical Theory of Communication by Claude Shannon>[§7, Theorems 5–6 and discussion, printed p. 14; Appendix 3, printed p. 30: block and conditional entropy approximations]{ref} The fair marginal <bits> of the <binary Markov source> provide a small exact example of that distinction.
Back to article page
