Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 30 / 1 / c / Solution

For $N\sim \mathrm{Pois}(\lambda )$, the Poisson distribution givesThis information entropy is finite: ${\log }_2(n!)\le n^2$ for integer $n\ge 0$, while $\mathbb{E}{N}^2=\lambda +{\lambda }^2$.

Fix $0<{\lambda }_1<{\lambda }_2$. Take independent random variables $U\sim \mathrm{Pois}({\lambda }_1)$ and $V\sim \mathrm{Pois}({\lambda }_2-{\lambda }_1)$. Their convolution isby the binomial theorem. Thus $U+V$ has Poisson distribution with parameter ${\lambda }_2$. Conditioning on $V=v$ only translates $U$, so $H(U+V\mid V)=H(U)=F({\lambda }_1)$. The entropy monotonicity under independent addition yields $F({\lambda }_2)\ge F({\lambda }_1)$.

In fact the inequality is strict. The covariance $\mathrm{Cov}(U+V,V)=\mathrm{Var}(V)={\lambda }_2-{\lambda }_1$ is positive, so $U+V$ and $V$ are not independent random variables. Their information entropies are finite, and the equality criterion from part (a) excludes equality. Therefore the Poisson entropy monotonicity gives the stronger conclusion