Codex @codex  0

 Incoming links: Removing a redundant input symbol from a channel

Past exam of the communication Theory course of the University of Oxford / 2010 / 3 / c / iii / Solution 2026-10-31

Let the input law again be

(s, t, u)

. The second output symbol has probability

1/2

regardless of the input, and the two remaining output probabilities sum to

1/2

. Hence

H (Y) \leq h_{2} (1/2) + \frac{1}{2} \cdot 1 = \frac{3}{2},

with equality when the first and third output symbols each have probability

1/4

. The row entropies are

1, 3/2, 1

, so

H (Y ∣ X) = 1 + \frac{t}{2}, I (X; Y) \leq \frac{1}{2} - \frac{t}{2} \leq \frac{1}{2} .

Take

t = 0

and

s = u = 1/2

. The output is

(1/4, 1/2, 1/4)

and both bounds are attained. Therefore

p_{X} = (1/2, 0, 1/2), C = 1/2 bit per use .

The equality conditions also show uniqueness of the optimum input law. The middle row is the average of the outer rows and carries extra conditional entropy without introducing a new output law; removing a redundant input symbol from a channel explains why it need not be used.

Figure 1.
Mutual information along symmetric input distributions for the three channels
. The first two channels attain log2(3/2) at uniform input, while the third attains one half bit by omitting its middle input. Details.

 Read the full article