Past exam of the mathematics course of the University of Cambridge / 2026 / iii / Paper 224 / 3

For nonnegative numbers $a_i,b_i$, put $a={\sum }_i{a}_i$ and $b={\sum }_i{b}_i$. The log-sum inequality iswith $0\log (0/b)=0$ and the usual extended-value convention when a denominator vanishes. Equality holds precisely when $a_i/b_i$ is constant over the indices with $a_i>0$.

Solved by gpt-5.6-sol high.

Let $K(y\mid x)$ be a Markov kernel, and let $P_Y=PK$, $Q_Y=QK$ be the output probability distributions. Applying the log-sum inequality for each $y$ to $a_x=P(x)K(y\mid x)$ and $b_x=Q(x)K(y\mid x)$ givesSumming over $y$ and using ${\sum }_{y}K(y\mid x)=1$ yields the data processing inequality for relative entropy

Solved by gpt-5.6-sol high.

The elementary logarithm inequality ${\log }_{2}u\le ({\log }_{2}e)(u-1)$ givesSince ${\sum }_x(P(x)-Q(x))=0$,This proves $D(P\Vert Q)\le (\log e){\chi }^2(P\Vert Q)$, relating relative entropy to chi-squared divergence.

Solved by gpt-5.6-sol high.

Writing $P,Q$ for the laws of $X,Y$, respectively, and using Jensen inequality for the concave natural logarithm,This is the lower-bound half of the Gibbs variational principle for relative entropy.

Solved by gpt-5.6-sol high.

Let $Z={\mathbb{E}}_Q{e}^g$ and define the exponentially tilted probability mass functionFor this choice every ratio $e^{g(x)}Q(x)/P^{\ast }(x)$ equals $Z$, so equality holds in the Jensen inequality used in part (d). Henceand $P^{\ast }$ is the maximizer. This is the finite-alphabet Gibbs variational principle for relative entropy.

Solved by gpt-5.6-sol high.