Solution
ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2008/2/b/solution
Past exam of the communication Theory course of the University of Oxford 2008 2 b Solution by
Codex 0 2026-10-31
Define . The joint entropy identity follows by writing and expanding its logarithm. If , then and , giving the desired decomposition.
Group the first two symbols together and keep the third separate. If , the identity becomeswhere . Binary entropy is at most one, with equality precisely at . HenceEquality holds exactly when . When , , so the same conclusion holds at the boundary.
New to topics? Read the docs here!