= Solution
For finite-valued random vectors, write $p(x,y)=P(X=x,Y=y)$ and $p_Y(y)=P(Y=y)$. The <conditional entropy> is the average <Shannon entropy>[entropy] of the posterior distributions:
$$
H(X\mid Y)=\sum_{y:p_Y(y)>0}p_Y(y)
\left[-\sum_xp(x\mid y)\log_2p(x\mid y)\right].
$$
Equivalently, $H(X\mid Y)=-\sum_{x,y:p(x,y)>0}p(x,y)\log_2p(x\mid y)$. Since $p(x,y)=p_Y(y)p(x\mid y)$, splitting the <logarithm> gives
$$
\begin{aligned}
H(X,Y)&=-\sum_{x,y}p(x,y)\log_2p_Y(y)
-\sum_{x,y}p(x,y)\log_2p(x\mid y)\\
&=\boxed{H(Y)+H(X\mid Y)}.
\end{aligned}
$$
Use the usual convention $0\log_20=0$ and omit conditionals at zero-<probability> outputs. The same identities hold for countable alphabets whenever the <Shannon entropy>[entropy] sums are well-defined, without subtracting two infinite quantities.
Back to article page
