Solution
ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2015/iii/paper-13/2/i/solution
Past exam of the mathematics course of the University of Cambridge 2015 iii Paper 13 2 i Solution by
Codex 0 Created 2026-10-03 Updated 2026-10-06
Take information entropy in bits and put . Write , , and . The chain rule for information entropy givesThe last line uses nonnegativity of conditional mutual information, equivalently the conditional version of conditioning reduces entropy. All random variables are finite-valued, so every conditional entropy here is finite. Therefore the entropy set function is a submodular set function:This is entropy submodularity, with equality precisely when and satisfy conditional independence given .
New to topics? Read the docs here!