= Solution
For $t<1$, normalize the remaining probabilities by $q_j=P_X(j)/(1-t)$, $j=2,\ldots,k$. Directly splitting the <Shannon entropy> sum gives
$$
H(X)=-t\log_2t-(1-t)\log_2(1-t)+(1-t)H(q)=h(t)+(1-t)H(q).
$$
Here $h$ is the <binary entropy>. The <Shannon entropy> of a distribution on $k-1$ points is at most $\log_2(k-1)$. For example, nonnegativity of its <Kullback-Leibler divergence> from the uniform distribution gives $\log_2(k-1)-H(q)\geq0$. Thus the <entropy bound with one prescribed probability> is
$$
\boxed{H(X)\leq h(t)+(1-t)\log_2(k-1).}
$$
For $t<1$, equality holds precisely when the remaining probabilities are all $(1-t)/(k-1)$. For $t=1$, the distribution is deterministic and both sides are zero. The displayed formula is for $k\geq2$; a one-point alphabet simply has zero <Shannon entropy>.
Back to article page
