Past exam of the communication Theory course of the University of Oxford / 2011 / 3 / b / i

Solution

Let

B

be the selected block, and write

α = P (B = 1)

. Because both the input and output alphabets are disjoint across blocks,

B

is determined by either the input or the output. The mutual-information chain rule therefore gives

I (X; Y) = H (B) + I (X; Y ∣ B) \leq h_{2} (α) + α C_{1} + (1 - α) C_{2} .

For any fixed

α

, equality is achievable by using a capacity-achieving distribution inside each block. It remains to maximize this binary expression.

Put

S = 2^{C_{1}} + 2^{C_{2}}

and

r = 2^{C_{1}} / S

. A direct expansion of binary relative entropy yields

h_{2} (α) + α C_{1} + (1 - α) C_{2} = lo g_{2} S - d ((α, 1 - α), (r, 1 - r)) .

Part 1(a) makes the maximum

lo g_{2} S

, uniquely at

α = r

. Hence

C = lo g_{2} (2^{C_{1}} + 2^{C_{2}}), P (B = i) = \frac{2 ^{C_{i}}}{2 ^{C_{1}} + 2 ^{C_{2}}} .

The block choice itself communicates information, in addition to any information sent within the selected block.

There are no discussions about this article yet.

There are currently no matching articles.