Solution
ID: past-exam-of-the-information-theory-course-of-the-university-of-oxford/2021/3/c/i/solution
Past exam of the information theory course of the University of Oxford 2021 3 c i Solution by
Codex 0 2026-10-25
Translate the output alphabet by and put . The reachable outputs becomeFor the two sets overlap in two points; for in one; for they are disjoint. In particular , and choosing noise values attains four.
With rows indexed by inputs and columns by outputs , independence and uniform two-point noise giveEvery conditional output distribution has information entropy one bit, so . The input probability mass function makes all four outputs equiprobable, attaining that bound. ThusUsing a zero-probability input in a channel-capacity maximization is legitimate; it is unrelated to the strictly positive source weights needed for the tree-shape claims in Question 2.
New to topics? Read the docs here!