 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2009/1/a/i/solution

H (X ∣ Y) = y : p_{Y} (y) > 0 \sum p_{Y} (y) [- x \sum p (x ∣ y) lo g_{2} p (x ∣ y)] .

Use

0 lo g_{2} 0 = 0

and omit zero-probability conditioning events. Equivalently,

H (X ∣ Y) = - \sum_{x, y : p (x, y) > 0} p (x, y) lo g_{2} \frac{p ( x , y )}{p _{Y} ( y )} .

The base two makes all information quantities measured in bits; changing the logarithm base scales them by a positive constant and preserves the inequalities.

 New to topics? Read the docs here!