Dyadic source matched to a complete prefix code
ID: dyadic-source-matched-to-a-complete-prefix-code
For a complete binary prefix code with lengths , the Kraft inequality is an equality: . Put . The difference between expected codeword length and Shannon entropy isThe Kullback-Leibler divergence is nonnegative and vanishes exactly when . Hence a complete code is entropy-tight exactly for its dyadic matching source. For example, lengths match probabilities . Being an optimal prefix code among symbol-by-symbol codes is a weaker condition and does not imply this equality.
New to topics? Read the docs here!