Finite cross entropy bounds countable information entropy ($H(p)\le -{\sum }_i{p}_i{\log }_2{g}_i$)

Let $p$ and $g$ be probability mass functions on a countable alphabet, with $g_i>0$ whenever $p_i>0$. If the cross-entropy $C=-{\sum }_i{p}_i{\log }_2{g}_i$ is finite, then $H(p)$ is finite, $H(p)\le C$, and $D(p\Vert g)=C-H(p)$. To avoid assuming the desired finiteness, first apply $\ln u\le u-1$ with $u=g_i/p_i$ to each positive $p_i$. For a finite initial segment,Zero coordinates satisfy this inequality too. Monotone convergence of the nonnegative information entropy summands and convergence of both total masses give $H(p)\le C<\infty$. Both logarithmic sums are then finite, justifying their difference and the relative entropy identity. This supplies a rigorous route to the maximum entropy distribution on the nonnegative integers without presupposing finite information entropy for every competing law.