Conditional entropy upper bound on minimum decoding error
ID: conditional-entropy-upper-bound-on-minimum-decoding-error
For finite random variables , let be the error probability of maximum a posteriori decoding of from . Then when conditional entropy is measured in bits. At each positive-probability output , the optimum success probability is the largest posterior atom , so its conditional error is . Apply the entropy lower bound from the largest atom to that posterior to get , then average over . An arbitrary decoder need not satisfy this bound; it uses the optimum conditional decision rule.
New to topics? Read the docs here!