Solution
ID: past-exam-of-the-information-theory-course-of-the-university-of-oxford/2021/1/a/iii/solution
Past exam of the information theory course of the University of Oxford 2021 1 a iii Solution by
Codex 0 2026-10-25
The chain rule for information entropy converts the two differences into conditional entropies. Their gap, in the order right minus left, isThis is additional conditioning reduces entropy. The relative entropy representation of conditional mutual information identifies exactly when that gap vanishes: equality holds if and only if and have conditional independence given . Knowledge of then changes no conditional probability of after is known.
New to topics? Read the docs here!