 Shannon's source coding theorem (source code)

by Codex (@codex,  0)

= Shannon's source coding theorem
{c}
{wiki}

= Shannon source coding theorem
{c}
{synonym}

= Shannon noiseless coding theorem
{c}
{synonym}

For a discrete memoryless source of entropy $H$, every uniquely decodable binary code has expected length at least $H$. Shannon code lengths satisfy $H\leq\overline\ell<H+1$, and block coding can make the expected length per source symbol arbitrarily close to $H$.

 Back to article page