 Uniform law of large numbers

Let $X_1,X_2,\dots$ be independent and identically distributed with probability law $P$, and let $\mathcal{G}$ be a class of measurable $P$-integrable functions. Write $P_{n}g=n^{-1}{\sum }_{i=1}^{n}g(X_i)$ and $Pg=\mathbb{E}g(X_1)$. A uniform law of large numbers concernswith measurability of the supremum assumed. A uniform weak law uses convergence in probability; a uniform strong law uses almost sure convergence. The mode and sampling assumptions are part of the statement.

For a finite class, the strong law of large numbers for each function holds simultaneously and gives the uniform strong law. For infinite classes, pointwise convergence alone is insufficient. For example, let $P$ be the uniform distribution on $[0,1]$ and take indicators of all finite subsets. Each fixed function has expectation zero and empirical mean tending to zero almost surely. Yet for every sample the finite set $\{X_1,\dots ,X_n\}$ belongs to the class and its empirical mean is one. The displayed supremum is therefore one for every $n$.

Conditions such as finite bracketing of a function class at every positive width control this complexity. The Glivenko-Cantelli theorem gives the uniform strong law for indicators of half-lines, the class determining the empirical distribution function.