= Solution
Because $Z$ is determined by $X,Y$, its <conditional entropy> satisfies $H(Z\mid X,Y)=0$. Equate the two forms of the <chain rule for conditional entropy> to obtain
$$
H(X\mid Y)=H(Z\mid Y)+H(X\mid Z,Y).
$$
Conditioning cannot increase classical <Shannon entropy>: $H(Z)-H(Z\mid Y)=I(Z:Y)\geq0$ by nonnegativity of <mutual information>. Thus $H(Z\mid Y)\leq H(Z)=h(p_e)$. When $Z=0$, the value of $X$ is exactly $f(Y)$ and $H(X\mid Y,Z=0)=0$. When $Z=1$ and $Y=y$, the value $f(y)$ is excluded, leaving at most $|J_X|-1$ possibilities. By <maximum entropy on a finite alphabet>,
$$
H(X\mid Y,Z=1)\leq\log_2(|J_X|-1).
$$
Averaging the two conditional cases now yields
$$
H(X\mid Z,Y)\leq p_e\log_2(|J_X|-1),
$$
and consequently
$$
\boxed{H(X\mid Y)\leq h(p_e)+p_e\log_2(|J_X|-1).}
$$
This is <Fano's inequality via an error indicator>. Events of zero probability contribute zero to the average and need no conditional distribution. For $|J_X|=1$, the inference is automatically correct and the entropy is zero; the displayed logarithmic form is intended for $|J_X|\geq2$. Optimality of the guess is not needed for the inequality: it holds for every deterministic $f$.
Back to article page
