The binary entropy function is the information entropy of a Bernoulli random variable:with endpoint values zero. It is symmetric under p↦1−p, reaches one bit at p=1/2, and hasin the interior. Its strict concavity expresses why an even mixture of two outcomes has more uncertainty than a strong bias toward either one. The curve in mutual information also shows how this same uncertainty becomes the loss term in binary symmetric channel capacity.
h2(p)=−plog2p−(1−p)log2(1−p),0≤p≤1,
h2′(p)=log2p1−p,h2′′(p)=−(ln2)p(1−p)1<0
Ancestors (6)
Incoming links (59)
- An observation can increase entropy
- Bernoulli typical-set example
- Binary cross-entropy
- Binary entropy product inequality
- Binary rate-distortion function
- Binary symmetric channel capacity
- Bit-position information in a binary channel
- Collective advantage over independent two-qubit filtering
- Correlated binary variables
- Data processing inequality
- Entropy bound for a binary mixture
- Entropy bound with one prescribed probability
- Entropy function for a q-ary alphabet
- Entropy lower bound from the largest atom
- Entropy of a binary diagonal qubit mixture
- Entropy rate and coding length
- Fano's inequality
- Fano's inequality via an error indicator
- Information entropy dominates min-entropy
- Input distribution optimization for a binary symmetric channel
- Joint entropy
- Mutual information
- Noisy parity total correlation
- One-shot classical-quantum coding converse
- Past exam of the communication Theory course of the University of Oxford / 2008 / 2 / b / Solution
- Past exam of the communication Theory course of the University of Oxford / 2010 / 2 / b / i / Solution
- Past exam of the information theory course of the University of Oxford / 2023 / 3 / a / Solution
- Past exam of the information theory course of the University of Oxford / 2023 / 3 / b / Solution
- Past exam of the laser science and quantum information processing course of the University of Oxford / 2006 / 7 / 3 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2003 / iii / Paper 35 / 3 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2005 / iii / Paper 37 / 4 / a / Solution
- Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 30 / 3 / a / Solution
- Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 30 / 3 / c / Solution
- Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 33 / 3 / c / Solution
- Past exam of the mathematics course of the University of Cambridge / 2007 / iii / Paper 31 / 3 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2007 / iii / Paper 34 / 3 / d / Solution
- Past exam of the mathematics course of the University of Cambridge / 2007 / iii / Paper 73 / 1 / d / Solution
- Past exam of the mathematics course of the University of Cambridge / 2007 / ii / Paper 2 / 11G / Solution
- Past exam of the mathematics course of the University of Cambridge / 2008 / iii / Paper 58 / 1 / ii / Solution
- Past exam of the mathematics course of the University of Cambridge / 2008 / iii / Paper 58 / 1 / i / Solution
- Past exam of the mathematics course of the University of Cambridge / 2008 / iii / Paper 60 / 1 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2010 / iii / Paper 51 / 3 / d / Solution
- Past exam of the mathematics course of the University of Cambridge / 2010 / iii / Paper 51 / 3 / e / Solution
- Past exam of the mathematics course of the University of Cambridge / 2012 / iii / Paper 65 / 4 / i / 1 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2013 / iii / Paper 12 / 4 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2014 / iii / Paper 60 / 2 / ii / a / Solution
- Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 66 / 4 / iii / Solution
- Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 66 / 5 / iii / Solution
- Past exam of the mathematics course of the University of Cambridge / 2015 / iii / Paper 66 / 5 / i / Solution
- Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 323 / 5 / 1 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2016 / iii / Paper 323 / 5 / 3 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 108 / 4 / Solution
- Past exam of the mathematics course of the University of Cambridge / 2021 / ii / Paper 2 / 3K / Solution
- Past exam of the mathematics course of the University of Cambridge / 2024 / ii / Paper 1 / 11K / b / Solution
- Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 323 / 3 / ii / b / Solution
- Past exam of the mathematics course of the University of Cambridge / 2025 / ii / Paper 2 / 12K / Solution
- Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 323 / 4 / c / iii / Solution
- Rate-distortion function
- Redundancy of a binary prefix code
Discussion (0) New discussion
There are no discussions about this article yet.
Articles by others on the same topic (0)
There are currently no matching articles.