 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2009/3/a/solution

For finite-valued variables, mutual information is

I (X; Y) = x, y \sum p (x, y) lo g_{2} \frac{p ( x , y )}{p _{X} ( x ) p _{Y} ( y )} = H (Y) - H (Y ∣ X) .

W (y ∣ x)

on finite alphabets, and conditional on an input sequence its outputs are independent with the product law

\prod_{i} W (y_{i} ∣ x_{i})

C = P_{X} max I (X; Y)

bits per channel use. The maximum is over all input distributions, with the transition law held fixed.

 New to topics? Read the docs here!