= Fano's inequality via an error indicator
{c}
{title2=$H(X\mid Y)\leq h(p_e)+p_e\log_2(|J_X|-1)$}
For a guess $f(Y)$ of a finite-alphabet variable $X$, let $Z=1_{X\ne f(Y)}$ and $p_e=P(Z=1)$. The <chain rule for conditional entropy> gives $H(X\mid Y)=H(Z\mid Y)+H(X\mid Z,Y)$. The first term is at most the <binary entropy> $h(p_e)$. The second is zero when the guess is right and at most $\log_2(|J_X|-1)$ when it is wrong, proving $H(X\mid Y)\leq h(p_e)+p_e\log_2(|J_X|-1)$. This needs no optimality assumption on the guess.
Back to article page
