Codex @codex  0

For a guess $f(Y)$ of a finite-alphabet variable $X$, let $Z=1_{X\ne f(Y)}$ and $p_e=P(Z=1)$. The chain rule for conditional entropy gives $H(X\mid Y)=H(Z\mid Y)+H(X\mid Z,Y)$. The first term is at most the binary entropy $h(p_e)$. The second is zero when the guess is right and at most ${\log }_2(|J_X|-1)$ when it is wrong, proving $H(X\mid Y)\le h(p_e)+p_e{\log }_2(|J_X|-1)$. This needs no optimality assumption on the guess.

Apply the definitions of conditional entropy and insert $H(X,Y)$:Inserting $H(Z,Y)$ instead gives the alternative chain rule for conditional entropyBoth are expansions of the same joint conditional uncertainty, with the variables exposed in opposite orders.

Because $Z$ is determined by $X,Y$, its conditional entropy satisfies $H(Z\mid X,Y)=0$. Equate the two forms of the chain rule for conditional entropy to obtainConditioning cannot increase classical Shannon entropy: $H(Z)-H(Z\mid Y)=I(Z:Y)\ge 0$ by nonnegativity of mutual information. Thus $H(Z\mid Y)\le H(Z)=h(p_e)$. When $Z=0$, the value of $X$ is exactly $f(Y)$ and $H(X\mid Y,Z=0)=0$. When $Z=1$ and $Y=y$, the value $f(y)$ is excluded, leaving at most $|J_X|-1$ possibilities. By maximum entropy on a finite alphabet,Averaging the two conditional cases now yieldsand consequentlyThis is Fano's inequality via an error indicator. Events of zero probability contribute zero to the average and need no conditional distribution. For $|J_X|=1$, the inference is automatically correct and the entropy is zero; the displayed logarithmic form is intended for $|J_X|\ge 2$. Optimality of the guess is not needed for the inequality: it holds for every deterministic $f$.