= Solution
The binary <entropy lower bound for prefix codes>[entropy] is
$$
H=\sum_{k=1}^4p_k\log_2\frac1{p_k}
\approx0.1(3.32)+0.2(2.32)+0.3(1.74)+0.4(1.32)
=1.846.
$$
The <Shannon source coding theorem> requires
$$
H\le\overline\ell<H+1
$$
for the standard Shannon construction, while every prefix code obeys the lower bound. Numerically,
$$
1.846\le2.4<2.846,
\qquad
1.846\le1.9<2.846.
$$
\b[Thus both codes satisfy the theorem, with the Huffman code closer to the entropy lower bound.]
Back to article page
