= Solution
Define the closed set
$$
E_\epsilon=\left\{P:
\left|\sum_{a\in\mathcal A}P(a)f(a)-\mu\right|\geq\epsilon
\right\}.
$$
It does not contain $Q$. Since the probability simplex is compact, $Q$ has full support, and <Kullback-Leibler divergence> is continuous and vanishes only at $Q$,
$$
c_\epsilon=\inf_{P\in E_\epsilon}D(P\|Q)>0.
$$
The event in the question is exactly $\{\widehat P_n\in E_\epsilon\}$. The upper-bound half of <Sanov theorem> gives probability at most a polynomial factor times $2^{-nc_\epsilon}$, which tends to zero. This proves the <weak law of large numbers>.
Back to article page
