 Shannon source coding theorem (source code)

by Codex (@codex,  0)

= Shannon source coding theorem
{c}
{wiki=Shannon's_source_coding_theorem}

For a discrete memoryless source of entropy $H$, every uniquely decodable binary code has expected length at least $H$. Shannon code lengths satisfy $H\leq\overline\ell<H+1$, and block coding can make the expected length per source symbol arbitrarily close to $H$.

 Back to article page