Entropy bound from overlap with a pure state Created 2026-10-06 Updated 2026-10-07
If a density operator on an dimensional Hilbert space has overlap with a fixed pure state, then . Dephase in a basis containing and apply the entropy bound with one prescribed probability. The abstract state bound is attained by .
Information entropy dominates min-entropy Created 2026-09-28 Updated 2026-10-14
For a discrete random variable with probability mass function , using base-two logarithms for both Shannon entropy and min-entropy,
The first inequality says that average information content cannot be below the least information content of a possible outcome. For a finite alphabet, equality holds exactly when the probability mass function is uniform on its positive-probability outcomes. Impossible outcomes do not need to share that positive probability. In particular, has both quantities equal to bit, while has both zero. The same equality characterization holds for a countable alphabet: equality forces only finitely many positive-probability outcomes, because all must have the same positive mass. The uniform maximum of Shannon entropy is discussed in Shannon's original treatment.[ref]
For the proof, gives ; taking the expected value proves information entropy dominates min-entropy. More precisely, the nonnegative gap is
It vanishes precisely when every positive term vanishes, namely for every possible outcome. With such outcomes, normalization forces . If Shannon entropy is infinite, the inequality still holds and is strict, because min-entropy of a countable probability mass function is finite. No subtraction of two infinite quantities is needed.
An analytical family makes the gap visible. Give probability and each of probability , with . Then is a most likely outcome, so its probability is also the best single-guess success probability, and
where is binary entropy. The entropy bound with one prescribed probability explains the second expression: it separates whether occurred from which of four equally likely alternatives occurred. At , all five outcomes are uniform and both quantities are bits. At , Shannon entropy is bits while min-entropy is bit, reproducing the biased example in min-entropy. At , both are zero; the four impossible alternatives contribute nothing. For every , the probability mass function is nonuniform on its possible outcomes and the gap is strictly positive.
Figure 1.
An analytical five-outcome model separates average surprise from the surprise of the most likely outcome
. Details.
For finite Shannon entropy, exponentiating the inequality gives the useful but potentially loose bound . Equality holds exactly for a discrete uniform distribution on the possible outcomes. Thus Shannon entropy alone supplies a lower bound on optimal success, not the upper bound needed to promise that every guess is unlikely. Shannon's source coding theorem concerns average compression under its hypotheses, while min-entropy answers the single-guess question directly. The two quantities serve different tasks even when both are expressed in bits.
Choose an orthonormal basis of the joint -dimensional Hilbert space whose first vector is the given purification of a density operator . Apply rank-one dephasing to in this basis, and denote the resulting diagonal probabilities by . Their first entry is
The preceding entropy increase under nonselective projective measurement and entropy bound with one prescribed probability yield
Therefore the quantum Fano inequality is
The quantity is the entanglement fidelity of on ; it is already a squared overlap, so it is , rather than , that enters the binary entropy. The argument is an instance of the entropy bound from overlap with a pure state in dimension . At the output is the original pure state and its Von Neumann entropy is zero. For the system is trivial and the same zero-entropy conclusion holds without evaluating .
For , normalize the remaining probabilities by , . Directly splitting the Shannon entropy sum gives
Here is the binary entropy. The Shannon entropy of a distribution on points is at most . For example, nonnegativity of its Kullback-Leibler divergence from the uniform distribution gives . Thus the entropy bound with one prescribed probability is
For , equality holds precisely when the remaining probabilities are all . For , the distribution is deterministic and both sides are zero. The displayed formula is for ; a one-point alphabet simply has zero Shannon entropy.