Almost sure convergence ($X_n\stackrel{a.s.}{\to }X$)

A sequence of random variables converges almost surely when the set of outcomes on which pointwise convergence fails has probability zero.

A sequence converges completely to $X$ when ${\sum }_n\mathrm{Pr}(|X_n-X|>ϵ)<\infty$ for every $ϵ>0$. The first Borel-Cantelli lemma implies almost sure convergence. For independent $X_n$, any limit in probability is almost surely constant; the second Borel-Cantelli lemma then gives the converse from almost sure to complete convergence. Independence is essential: $X_n=1_{\{U\le 1/n\}}$ for one uniform random variable $U$ converges almost surely to zero but its exceedance probabilities have a divergent sum.

If independent random variables $X_n$ converge almost surely to a finite $X$, then ${\mathrm{limsup}}_n{X}_n$ is measurable for the tail sigma-algebra. The Kolmogorov zero-one law forces every rational threshold event for that limit to have probability zero or one. A real-valued probability distribution with this property is a point mass, so $X$ is almost surely constant. Consequently the Second Borel-Cantelli lemma upgrades almost sure convergence of independent sequences to complete convergence.

Random variables $X$ and $Y$ are almost surely equal when $\mathbb{P}(X=Y)=1$. Probability theory conventionally identifies random variables that differ only on a null event.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, and let $\mu =\mathbb{E}[X_1]$ be their expected value. Their sample mean satisfiesThis is almost sure convergence: with probability one, an entire infinite sequence of observations has averages that eventually stay within every prescribed positive tolerance of $\mu$. The variance need not be finite. The strong law of large numbers implies the weak law of large numbers, which asserts convergence in probability at each increasing sample size.

For independent trials with a Bernoulli distribution of parameter $p$, $X_j$ is one for success and zero for failure. Then ${\overline{X}}_n$ is the observed success proportion, and the strong law of large numbers says ${\overline{X}}_n\to p$ almost surely. This turns a theoretical probability into a quantity one can estimate by repetition. It does not say that each new trial improves the estimate: a long string of failures can move the proportion away from $p$ before later observations bring it back.

The strong law of large numbers gives the limiting location of the sample mean, while the central limit theorem describes its fluctuations when the variance is finite and positive. The two assertions can hold together:The first error shrinks to zero; the second has been enlarged enough to retain a nontrivial probability distribution. The second assertion is convergence in distribution, not a claim that the enlarged error along one sequence approaches a particular normal random variable.

The assumptions make a practical difference. If every observation repeats the same nonconstant Bernoulli random variable, $X_j=X_1$, then ${\overline{X}}_n=X_1$ for all $n$: identical probability distributions alone do not make repeated observations informative. For independent observations with a Standard Cauchy distribution, the Cauchy sample mean has an unchanged distribution, so it does not concentrate around a constant. Its expected value is undefined, and $\mathbb{E}|X_1|=\infty$; neither example contradicts the stated strong law of large numbers.

A brief proof outline in the case of finite variance uses the Kolmogorov maximal inequality for the centered partial sums. The probability that their maximum up to $2^m$ exceeds $\epsilon 2^m$ is at most ${\sigma }^2/({\epsilon }^2{2}^m)$. These bounds are summable, so the Borel-Cantelli lemma controls all sufficiently large such blocks and hence every intermediate sample size. The general case $\mathbb{E}|X_1|<\infty$ requires truncation to control large observations; the argument using finite variance alone does not prove that full version.

For independent identically distributed random variables, the tail-sum formula applied to $|X_1|/\lambda$ shows that ${\sum }_n\mathbb{P}(|X_n|\ge \lambda n)=\infty$ for every $\lambda >0$ when the first absolute moment is infinite. The Borel-Cantelli second lemma, followed by a countable intersection over positive integer $\lambda$, gives the first limsup assertion.

If $|S_n|/n$ had finite limsup along such a sample path, it would eventually be bounded by some finite constant $C$. Then $X_n=S_n-S_{n-1}$ would imply $|X_n|/n\le 2C$ eventually, a contradiction. This is a necessary-integrability counterpart to the strong law of large numbers; without a finite first absolute moment, the normalized sums cannot converge to any finite value.

Take independent random variables equal to $1+n$ and $1-n$ with probability $1/(2n\log n)$ each, and otherwise equal to one. Their first absolute deviations are $1/\log n\to 0$, so $X_n\to 1$ in L1 convergence. The same triangle-inequality argument gives convergence in $L^1$ of their arithmetic averages to one. For $p>1$, however, $\mathbb{E}|X_n-1{|}^p=n^{p-1}/\log n\to \infty$.

The exceptional jumps occur infinitely often almost surely by the Borel-Cantelli second lemma, since $\sum 1/(n\log n)=\infty$. If the arithmetic averages $S_n/n$ converged to any finite value along a sample path, then $X_{n+1}/n=S_n/n-[(n-1)/n][S_{n-1}/(n-1)]$ would tend to zero. At each exceptional jump its magnitude is at least one. Thus the averages have no finite almost sure limit. Independence and convergence of first-moment averages do not alone give a strong law of large numbers.

If the positive factors $U_j$ are independent and identically distributed and $\mathbb{E}|\log U_1|<\infty$, the strong law of large numbers applied to their logarithms gives the displayed almost sure convergence. A negative limit forces the products to tend to zero; a positive limit forces them to tend to infinity. A zero limit alone gives neither conclusion. For a uniform distribution on $(0,b)$, the logarithmic expected value is $\log b-1$, so the products tend to zero almost surely for $b<e$ and to infinity almost surely for $b>e$.

If ${\sup }_n\mathbb{E}|X_n{|}^2<\infty$ and the $X_n$ are pairwise uncorrelated random variables, then $n^{-1}{\sum }_{k=1}^n(X_k-\mathbb{E}{X}_k)\to 0$ almost surely. Independence is unnecessary for this version of the strong law of large numbers. A uniformly bounded martingale difference sequence meets its hypotheses, since the differences have mean zero and are orthogonal in $L^2$.

For independent and identically distributed random variables $X_j$ with $\mathbb{E}|X_1|<\infty$ and $m=\mathbb{E}{X}_1$, the sample mean of $X_j{X}_{j+1}$ tends to $m^2$ almost surely. Split into odd and even indexed products; each subsequence consists of independent and identically distributed random variables that are integrable random variables. Apply the strong law of large numbers separately.

If independent centered random variables $(Y_n)$ satisfy ${\sum }_n\mathrm{Var}(Y_n)<\infty$, then ${\sum }_n{Y}_n$ converges almost surely. It follows from Kolmogorov's maximal inequality and a Cauchy-tail argument.

For independent identically distributed centered variables of variance one under standard moment assumptions, the law of the iterated logarithm describes the almost-sure envelope of their partial sums on the scale $\sqrt{2n\log \log n}$.

For standard Brownian motion, the Brownian reflection principle bounds the probability that its maximum up to $a^n$ exceeds $(1+\epsilon )\sqrt{2a^n\log \log (a^n)}$ by $2(n\log a{)}^{-(1+\epsilon {)}^2}$. This is summable for every $a>1$ and $\epsilon >0$. The Borel-Cantelli first lemma controls these geometric times, monotonicity of the normalizing function controls the intervening times, and countable choices $a\downarrow 1$, $\epsilon \downarrow 0$ give the bound. Equality requires a separate lower-bound proof.

For a simple symmetric random walk, geometric blocking and the Borel-Cantelli first lemma yield the upper estimate ${\mathrm{limsup}}_n{S}_n/\sqrt{2n\log \log n}\le 1$ almost surely. Equality requires a separate lower-bound argument.

For a simple symmetric random walk, almost surely