= Solution
For a discrete <random variable> with probabilities $p_i$, its <Shannon entropy> in bits is $H(X)=-\sum_ip_i\log_2p_i$, with $0\log_20=0$. <Gibbs inequality> states that
$$
D(p\Vert q)=\sum_ip_i\log_2\frac{p_i}{q_i}\geq0,
$$
with equality exactly when $p=q$. If $q_i=0<p_i$, the left side is infinite. Otherwise apply $-\ln u\geq1-u$ to $u=q_i/p_i$ for $p_i>0$:
$$
(\ln2)D(p\Vert q)\geq\sum_{p_i>0}(p_i-q_i)\geq0.
$$
Equality requires $q_i=p_i$ on the support and no remaining mass outside it. In particular, comparison with the uniform <probability distribution> on two outcomes gives binary <Shannon entropy> at most $1$.
Write $q=1-p_1$. For $q>0$, grouping the last two outcomes gives
$$
H(p_1,p_2,p_3)=H(p_1,q)+qH(p_2/q,p_3/q)\leq H(p_1,q)+q.
$$
Equality holds if $p_2=p_3$; if $q=0$ it also holds, and then both are zero. Thus \b[equality holds exactly when $p_2=p_3$].
For the <discrete memoryless channel>, assume $\alpha,\beta\geq0$ and $\alpha+\beta\leq1$, and set $w=1-\alpha-\beta$. Both conditional output <probability distributions> have <Shannon entropy> $H(w,\alpha,\beta)$. The third output has <probability> $\beta$ regardless of the input; the first two have total <probability> $1-\beta$. Their <Shannon entropy> is maximized by making them equal, which a uniform input achieves. Hence the <channel capacity> is
$$
C=H\left(\frac{1-\beta}{2},\frac{1-\beta}{2},\beta\right)-H(w,\alpha,\beta)
$$
and therefore
$$
\boxed{C=(1-\beta)\bigl[1-\log_2(1-\beta)\bigr]+w\log_2w+\alpha\log_2\alpha.}
$$
The usual zero-times-log convention includes $\beta=1$, when $C=0$. If $w=\alpha$, the rows coincide and every input achieves zero <mutual information>; otherwise the uniform input is the unique maximizing input.
Back to article page
