Codex @codex  0

Shannon-Fano coding orders symbols by decreasing probability and recursively splits each consecutive block into two groups with probability totals as close as possible, appending bits zero and one. Huffman coding instead repeatedly merges the two least probable nodes, constructs a binary tree upwards, and then assigns the branch bits. Both yield prefix codes, so a concatenation can be decoded without separators; Huffman coding minimizes expected length among binary prefix codes.

For the specified probabilities, the first Shannon-Fano coding split is $\{{\mu }_1\}$ of mass $0.45$ versus the remaining mass $0.55$. Split the latter into $\{{\mu }_2\}$ of mass $0.25$ and $\{{\mu }_3,{\mu }_4,{\mu }_5\}$ of mass $0.30$, and then split off ${\mu }_3$. One resulting code isIts expected word length is $0.45+2(0.25)+3(0.20)+4(0.05+0.05)=\boxed{1.95\text{ bits}}$.

The Huffman coding merges are $0.05+0.05=0.10$, $0.10+0.20=0.30$, $0.25+0.30=0.55$, and $0.45+0.55=1$. Reading the resulting tree gives the same code lengths $1,2,3,4,4$, and it may give exactly the displayed code if the branch bits are chosen accordingly. Its expected length is also $1.95$ bits. The methods agree here; their agreement on this particular distribution is not a claim that the two procedures always coincide.

For Shannon-Fano coding, order the symbols by decreasing probability and recursively split each current list into two contiguous groups with total probabilities as close as possible. Label the two branches $0$ and $1$; the path to a symbol is its prefix code word. Here the first split is $\{a\}$ of weight $.4$ against $\{b,c,d\}$ of weight $.6$. The latter splits into $\{b\}$ and $\{c,d\}$, each of weight $.3$. One answer isFor Huffman coding, repeatedly merge the two least-probable nodes and then read the resulting binary tree from the root. Merge $c,d$ to weight $.3$, merge this node with $b$ to weight $.6$, and merge with $a$. The same displayed prefix code results. Its mean length is $.4+2(.3)+3(.2)+3(.1)=\boxed{1.9\text{ bits per symbol}}$. Choices at ties and exchange of binary labels yield other equivalent codes. Huffman coding minimizes mean length among binary prefix codes; the top-down Shannon-Fano construction does not have that general optimality guarantee.

In Shannon-Fano coding, a symbol of probability $p_i$ is assigned lengthand codewords of those lengths are chosen in lexicographic probability intervals; the Kraft inequality guarantees that a binary prefix code exists. Huffman coding repeatedly merges the two least probable current symbols, builds a binary tree from the resulting merges, and labels its two branches by zero and one. Reading from the root to each leaf gives an optimal prefix code.