Past exam of the mathematics course of the University of Cambridge / 2026 / ii / Paper 2 / 3H (Coding & Cryptography)

The source is Bernoulli when the $X_n$ are independent and identically distributed with a common law $p$ on the finite alphabet.

It is reliably encodable at rate $r$ if there are block encoders into at most $2^{nr}$ codewords and decoders whose block error probability tends to zero as $n\to \infty$.

The information rate is ${\lim }_{n\to \infty }{n}^{-1}H(X_1,\dots ,X_n)$ when the limit exists.

The asymptotic equipartition property is $-n^{-1}{\log }_{2}p(X_1,\dots ,X_n)\to H$ in probability. For an i.i.d. source this is the weak law applied to $-{\log }_{2}p(X_i)$, and independence gives $H(X_1,\dots ,X_n)=nH(X)$. Thus its information rate and AEP constant are both $H(X)$.