Past exam of the communication Theory course of the University of Oxford / 2008 / 2 / b / Solution

Define $H(X\mid Y)={\sum }_y\mathbb{P}(Y=y)H(X\mid Y=y)$. The joint entropy identity $H(X,Y)=H(Y)+H(X\mid Y)$ follows by writing $p_{xy}=q_y{p}_{x\mid y}$ and expanding its logarithm. If $Y=g(X)$, then $H(Y\mid X)=0$ and $H(X,Y)=H(X)$, giving the desired decomposition.

Group the first two symbols together and keep the third separate. If $c<1$, the identity becomeswhere $h(t)=-t{\log }_{2}t-(1-t){\log }_2(1-t)$. Binary entropy is at most one, with equality precisely at $1/2$. HenceEquality holds exactly when $a=b=(1-c)/2$. When $c=1$, $a=b=0$, so the same conclusion holds at the boundary.