Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2008/1/a/ii/solution

The entropy chain rule gives
Conditional entropy is nonnegative, so . For finite entropies, if , equality holds exactly when , meaning is a function of almost surely. If , interchange the variables. When their entropies are equal, equality requires both functional relations. Zero conditional entropy has this characterization because each conditional probability distribution must be a point mass.

New to topics? Read the docs here!