Solution

ID: past-exam-of-the-mathematics-course-of-the-university-of-cambridge/2024/iii/paper-224/3/b/solution

Define the closed set
It does not contain . Since the probability simplex is compact, has full support, and Kullback-Leibler divergence is continuous and vanishes only at ,
The event in the question is exactly . The upper-bound half of Sanov theorem gives probability at most a polynomial factor times , which tends to zero. This proves the weak law of large numbers.

New to topics? Read the docs here!