Past exam of the mathematics course of the University of Cambridge / 2025 / iii / Paper 224 / 2

Let $X_1,X_2,\dots$ be i.i.d. with mass function $Q$ on a finite alphabet, and let ${\hat{P}}_n$ be their empirical distribution. Sanov theorem states that for every set $\Gamma$ of probability mass functions,andwhere interior and closure use the probability-simplex topology. Thus the empirical distributions satisfy a large deviation principle with rate function $D_e(\cdot \Vert Q)$.

The likelihood-ratio event depends only on the type $R={\hat{P}}_n$ and isequivalently $D(R\Vert Q)\ge D(R\Vert P)$. This constraint defines a closed subset of the finite probability simplex, so Sanov's upper bound gives the exponentIt is strictly positive: the only distribution with zero divergence from $Q$ is $R=Q$, but $Q$ violates the constraint because $0=D(Q\Vert Q)<D(Q\Vert P)$. Compactness and continuity under full support keep the infimum away from zero. This exponent is the Chernoff information between $P$ and $Q$.

An error implies that some $\theta \ne {\theta }^{\ast }$ has likelihood at least that of ${\theta }^{\ast }$. A finite union bound and part (b) therefore givewhereEvery inner infimum is strictly positive by the argument in part (b), and the minimum of finitely many positive numbers is positive.