Write and for the expectation and empirical measure. A sufficient bracketing of a function class condition is that, for every , finitely many brackets cover , with measurable integrable endpoints and . The uniform strong law from finite L1 bracketing then gives
Here the observations are independent and identically distributed. For an uncountable class, one either assumes a measurable supremum, for example through a pointwise separable function class, or formulates the conclusion as a pathwise bound on a common probability-one event.
For the parameterized class, the Heine-Borel theorem makes compact. Put . A countable dense subset of gives the same supremum because of continuity, so is measurable. Define the modulus
The supremum is measurable by taking a countable dense subset of the compact set of admissible pairs. For each , uniform continuity on gives . Also , and . Thus the dominated convergence theorem gives .
Choose a finite net of radius in . The lower and upper envelopes of over each closed ball are measurable integrable function brackets; the same countable-dense-set argument applies within each such compact ball. Their widths are at most . They cover the whole class, so its bracketing of a function class condition follows. Moreover, dominated convergence shows that is continuous. Both empirical and population functions therefore have a supremum over a common countable dense parameter set. Applying the uniform strong law from finite L1 bracketing proves uniform almost-sure convergence over the entire compact parameter set, rather than merely convergence at each fixed parameter.
For the exponential family,
where and . Thus a weak sufficient condition is bounded on , almost surely, and integrability of under the sampling law. No positive lower bound on over the whole real line is required. Zeros off the sampling support are harmless: choose arbitrary finite versions of the log-densities on that common null set when applying the function-class theorem.
If the sampling law is , one convenient assumption is that is a compact subset of the interior of the finite domain of the cumulant function of an exponential family, and
On that interior, is continuous, hence bounded on . Finiteness of at for some implies , hence . The displayed integral gives the remaining integrability. If the observations come from an arbitrary unrelated law, assumptions on and alone cannot control that law's tails; the sampling-law integrability must then be stated explicitly.
Uniform law of large numbers Created 2026-09-24 Updated 2026-11-05
Let be independent and identically distributed with probability law , and let be a class of measurable -integrable functions. Write and . A uniform law of large numbers concerns
with measurability of the supremum assumed. A uniform weak law uses convergence in probability; a uniform strong law uses almost sure convergence. The mode and sampling assumptions are part of the statement.
For a finite class, the strong law of large numbers for each function holds simultaneously and gives the uniform strong law. For infinite classes, pointwise convergence alone is insufficient. For example, let be the uniform distribution on and take indicators of all finite subsets. Each fixed function has expectation zero and empirical mean tending to zero almost surely. Yet for every sample the finite set belongs to the class and its empirical mean is one. The displayed supremum is therefore one for every .
Conditions such as finite bracketing of a function class at every positive width control this complexity. The Glivenko-Cantelli theorem gives the uniform strong law for indicators of half-lines, the class determining the empirical distribution function.