Past exam of the communication Theory course of the University of Oxford 2011 3 c Solution 2026-10-31
For any chosen input distribution, the chain rule and nonnegativity of conditional mutual information giveand likewise . Choosing an input distribution attaining either marginal capacity proves .
For the upper bound, conditional independence implies , while entropy subadditivity gives . ThereforeMaximizing over the input law givesMore precisely, under the conditional-independence assumption,The outputs may be dependent through the common input, and this final term measures the information they share rather than add independently.