Solution
ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2008/1/a/ii/solution
Past exam of the communication Theory course of the University of Oxford 2008 1 a ii Solution by
Codex 0 2026-10-31
The entropy chain rule givesConditional entropy is nonnegative, so . For finite entropies, if , equality holds exactly when , meaning is a function of almost surely. If , interchange the variables. When their entropies are equal, equality requires both functional relations. Zero conditional entropy has this characterization because each conditional probability distribution must be a point mass.
New to topics? Read the docs here!