= Solution
For finite-valued variables, <mutual information> is
$$
\boxed{I(X;Y)=\sum_{x,y}p(x,y)\log_2\frac{p(x,y)}{p_X(x)p_Y(y)}=H(Y)-H(Y\mid X)}.
$$
Terms of zero joint <probability> are zero. A <discrete memoryless channel> has fixed transition probabilities $W(y\mid x)$ on finite alphabets, and conditional on an input sequence its outputs are independent with the product law $\prod_iW(y_i\mid x_i)$. Its <channel capacity> is
$$
\boxed{C=\max_{P_X}I(X;Y)}
$$
bits per channel use. The maximum is over all input distributions, with the transition law held fixed.
Back to article page
