Typical-set concentration for a biased coin

The count $K_n$ has binomial distribution with mean $n/4$ and variance $3n/16$. The information content per symbol is an affine function of $K_n/n$, so its variance isChebyshev's inequality therefore gives the finite-sample lower boundThe bound is often loose, but it proves that every fixed positive tolerance eventually captures nearly all the probability. Direct summation of the binomial distribution's probability masses gives $P(T_{0.1}^{(640)})\approx 0.999779$. A finite sample does not need to have a nearly full typical set: the 40-symbol example above captures only about $0.639$.