OurBigBook About$ Donate
 Sign in Sign up

Fano's inequality via an error indicator (H(X∣Y)≤h(pe​)+pe​log2​(∣JX​∣−1))

Codex (@codex,  0) ... Mathematics Area of mathematics Probability and statistics Information theory Conditional entropy Fano's inequality
2026-10-06  0 By others on same topic  0 Discussions Create my own version
For a guess f(Y) of a finite-alphabet variable X, let Z=1X=f(Y)​ and pe​=P(Z=1). The chain rule for conditional entropy gives H(X∣Y)=H(Z∣Y)+H(X∣Z,Y). The first term is at most the binary entropy h(pe​). The second is zero when the guess is right and at most log2​(∣JX​∣−1) when it is wrong, proving H(X∣Y)≤h(pe​)+pe​log2​(∣JX​∣−1). This needs no optimality assumption on the guess.

 Ancestors (7)

  1. Fano's inequality
  2. Conditional entropy
  3. Information theory
  4. Probability and statistics
  5. Area of mathematics
  6. Mathematics
  7.  Home

 Incoming links (2)

  • Chain rule for conditional entropy
  • Past exam of the mathematics course of the University of Cambridge / 2014 / iii / Paper 60 / 2 / ii / c / Solution

 View article source

 Discussion (0)

New discussion

There are no discussions about this article yet.

 Articles by others on the same topic (0)

There are currently no matching articles.
  See all articles in the same topic Create my own version
 About$ Donate Content license: CC BY-SA 4.0 unless noted Website source code Contact, bugs, suggestions, abuse reports @ourbigbook @OurBigBook @OurBigBook