Past exam of the mathematics course of the University of Cambridge / 2024 / iii / Paper 224 / 3 / b / Solution

Define the closed setIt does not contain $Q$. Since the probability simplex is compact, $Q$ has full support, and Kullback-Leibler divergence is continuous and vanishes only at $Q$,The event in the question is exactly $\{{\hat{P}}_n\in E_{ϵ}\}$. The upper-bound half of Sanov theorem gives probability at most a polynomial factor times $2^{-n{c}_{ϵ}}$, which tends to zero. This proves the weak law of large numbers.