Law of large numbers 2026-10-14
Laws of large numbers give conditions under which sample means approach a deterministic expected value. For independent and identically distributed random variables with , the strong law of large numbers gives almost sure convergence, while the weak law of large numbers gives convergence in probability. These conclusions specify different modes of convergence of random variables, rather than a universal numerical accuracy guarantee. The central limit theorem is a separate result about the remaining fluctuations after rescaling.
The definition of convergence in distribution gives at each continuity point of . We must handle a moving argument rather than assume this convergence is uniform.
A cumulative distribution function has at most countably many discontinuities: each is a jump of positive size, and there are at most jumps of size at least , since their sizes sum to at most one. Thus we can choose continuity points with and . For fixed , eventually . The monotonicity of each cumulative distribution function implies
First let , using the definition of convergence in distribution at . Then let , using continuity at :
No result relating different convergence of random variables has been assumed.
Fix . Once , the following event inclusions hold outside :
Taking probabilities gives the explicit bounds
By the assumed convergence in probability, the error tends to zero. Therefore
The continuity of the cumulative distribution function at lets decrease to zero, proving This proves the needed implication directly, without invoking any theorem comparing convergence of random variables.