= Solution
Define $H(X\mid Y)=\sum_y\mathbb P(Y=y)H(X\mid Y=y)$. The joint <Shannon entropy>[entropy] identity $H(X,Y)=H(Y)+H(X\mid Y)$ follows by writing $p_{xy}=q_yp_{x\mid y}$ and expanding its logarithm. If $Y=g(X)$, then $H(Y\mid X)=0$ and $H(X,Y)=H(X)$, giving the desired decomposition.
<Group> the first two symbols together and keep the third separate. If $c<1$, the identity becomes
$$
H(a,b,c)=h(c)+(1-c)h\!\left(\frac a{1-c}\right),
$$
where $h(t)=-t\log_2t-(1-t)\log_2(1-t)$. <Binary entropy> is at most one, with equality precisely at $1/2$. Hence
$$
H(a,b,c)\le h(c)+1-c=H((1-c)/2,(1-c)/2,c).
$$
Equality holds exactly when $a=b=(1-c)/2$. When $c=1$, $a=b=0$, so the same conclusion holds at the boundary.
Back to article page
