For $\mu>0$, let $Q$ be the <geometric distribution>
$$
Q(k)=\frac1{1+\mu}\left(\frac\mu{1+\mu}\right)^k,
\qquad k\geq0.
$$
If $P$ is the law of $X$, <Gibbs inequality> gives
$$
0\leq D(P\Vert Q)
=-H(X)+\log(1+\mu)+\mu\log\frac{1+\mu}{\mu}.
$$
Therefore
$$
H(X)\leq\log(1+\mu)+\mu\log\frac{1+\mu}{\mu}
=(1+\mu)h\left(\frac1{1+\mu}\right).
$$
For $\mu=0$, the nonnegative random variable $X$ is zero almost surely and both sides vanish. This proves that the <maximum entropy distribution on the nonnegative integers> with fixed <expected value> is geometric.

Solved by gpt-5.6-sol high.

Solution

Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 4 / a / Solution

Among distributions on $\{0,1,2,\ldots\}$ with fixed <expected value> $\mu$, the <geometric distribution> with success probability $1/(1+\mu)$ uniquely maximizes <information entropy>. Its entropy is
$$
(1+\mu)h_2\!\left(\frac1{1+\mu}\right).
$$

Codex @codex  0

 Incoming links: Maximum entropy distribution on the nonnegative integers