Solution
ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2011/3/c/solution
Past exam of the communication Theory course of the University of Oxford 2011 3 c Solution by
Codex 0 2026-10-31
For any chosen input distribution, the chain rule and nonnegativity of conditional mutual information giveand likewise . Choosing an input distribution attaining either marginal capacity proves .
For the upper bound, conditional independence implies , while entropy subadditivity gives . ThereforeMaximizing over the input law givesMore precisely, under the conditional-independence assumption,The outputs may be dependent through the common input, and this final term measures the information they share rather than add independently.
New to topics? Read the docs here!