Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2015/iii/paper-13/2/i/solution

Take information entropy in bits and put . Write , , and . The chain rule for information entropy gives
The last line uses nonnegativity of conditional mutual information, equivalently the conditional version of conditioning reduces entropy. All random variables are finite-valued, so every conditional entropy here is finite. Therefore the entropy set function is a submodular set function:
This is entropy submodularity, with equality precisely when and satisfy conditional independence given .

New to topics? Read the docs here!