= Strong law of large numbers
{title2=$\overline X_n\xrightarrow{\mathrm{a.s.}}\mu$}
{wiki}
Let $X_1,X_2,\ldots$ be real <independent and identically distributed random variables> with $\mathbb E|X_1|<\infty$, and let $\mu=\mathbb E[X_1]$ be their <expected value>. Their <sample mean> satisfies
$$
\overline X_n=\frac1n\sum_{j=1}^nX_j\xrightarrow{\mathrm{a.s.}}\mu.
$$
This is <almost sure convergence>: with <probability> one, an entire infinite sequence of observations has averages that eventually stay within every prescribed positive tolerance of $\mu$. The <variance> need not be finite. The <strong law of large numbers> implies the <weak law of large numbers>, which asserts <convergence in probability> at each increasing sample size.
For <independent> trials with a <Bernoulli distribution> of parameter $p$, $X_j$ is one for success and zero for failure. Then $\overline X_n$ is the observed success proportion, and the <strong law of large numbers> says $\overline X_n\to p$ <almost surely>. This turns a theoretical <probability> into a quantity one can estimate by repetition. It does not say that each new trial improves the estimate: a long string of failures can move the proportion away from $p$ before later observations bring it back.
\Image[/law-of-large-numbers-averages.png]
{title=Bernoulli averages approach their mean: sample paths and exact error probabilities}
{description=The left panel shows four reproducibly simulated sequences of <independent> <Bernoulli trials> with success <probability> $p=0.2$. The right panel calculates $\mathbb P(|\overline X_n-p|\geq0.1)$ from the exact <binomial distribution> and compares it with the bound from the <Chebyshev inequality>. Small irregularities reflect integer-valued counts. The paths illustrate <almost sure convergence>; a finite simulation cannot prove it.}
The <strong law of large numbers> gives the limiting location of the <sample mean>, while the <central limit theorem> describes its fluctuations when the <variance> is finite and positive. The two assertions can hold together:
$$
\overline X_n\xrightarrow{\mathrm{a.s.}}\mu,
\qquad
\frac{\sqrt n(\overline X_n-\mu)}{\sigma}\xrightarrow{d}N(0,1).
$$
The first error shrinks to zero; the second has been enlarged enough to retain a nontrivial <probability distribution>. The second assertion is <convergence in distribution>, not a claim that the enlarged error along one sequence approaches a particular <normal random variable>.
The assumptions make a practical difference. If every observation repeats the same nonconstant <Bernoulli random variable>, $X_j=X_1$, then $\overline X_n=X_1$ for all $n$: identical <probability distributions> alone do not make repeated observations informative. For <independent> observations with a <Standard Cauchy distribution>, the <Cauchy sample mean has an unchanged distribution>, so it does not concentrate around a constant. Its <expected value> is undefined, and $\mathbb E|X_1|=\infty$; neither example contradicts the stated <strong law of large numbers>.
A brief proof outline in the case of finite <variance> uses the <Kolmogorov maximal inequality> for the centered partial sums. The <probability> that their maximum up to $2^m$ exceeds $\varepsilon2^m$ is at most $\sigma^2/(\varepsilon^2 2^m)$. These bounds are summable, so the <Borel-Cantelli lemma> controls all sufficiently large such blocks and hence every intermediate sample size. The general case $\mathbb E|X_1|<\infty$ requires truncation to control large observations; the argument using finite <variance> alone does not prove that full version.
Back to article page
