Information rate

The information rate is the limiting entropy per emitted symbol, when that limit exists.

A source is reliably encodable at rate $r$ if there are block encoders with at most $2^{nr}$ outputs and corresponding decoders whose probability of reconstructing the first $n$ source symbols incorrectly tends to zero as $n\to \infty$.

For a source $(X_n)$ and fixed positive integer $N$, form the block source $Y_i=(X_{(i-1)N+1},\dots ,X_{iN})$. Whenever the original information rate $h$ exists, the blocked source has information rate $Nh$, because