Mismatch penalty in lossless coding (${\mathbb{E}}_P[\ell (X)]=H(P,Q)$)

Suppose a binary prefix code has integer codeword lengths ${\ell }_i=-{\log }_2{q}_i$. Under a source with probability mass function $p_i$, its average codeword length is exactly cross-entropy, ${\sum }_i{p}_i{\ell }_i=H(P,Q)$. The Kraft inequality is satisfied because ${\sum }_i{2}^{-{\ell }_i}={\sum }_i{q}_i=1$. This exact interpretation requires suitable dyadic rational probabilities; a general $-{\log }_2{q}_i$ is not an integer codeword length. For positive $q_i$, a prefix code with lengths $\lceil -{\log }_2{q}_i\rceil$ exists by the Kraft inequality and has average length between $H(P,Q)$ and $H(P,Q)+1$. The ceiling function causes the one-symbol rounding overhead. The cross-entropy term describes mismatch overhead; those are distinct costs.