 Solution

ID: past-exam-of-the-communication-theory-course-of-the-university-of-oxford/2010/2/a/ii/solution

Z = g (X)

. Given

X

, the value of

Z

is certain, so

H (Z ∣ X) = 0

. Applying the entropy identity in the two orders gives

H (X) = H (X, Z) = H (Z) + H (X ∣ Z) \geq H (Z),

because conditional entropy is nonnegative. Consequently

H (g (X)) \leq H (X) .

A deterministic coarsening can merge distinguishable source outcomes, but cannot create additional Shannon entropy.

 New to topics? Read the docs here!