Convergence of random variables

The main modes include almost-sure convergence, convergence in probability, convergence in $L^p$, and convergence in distribution.

A sequence $Q_n$ is contiguous with respect to $P_n$ if $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$ for every measurable sequence of events. It transfers negligible-event assertions, including consistency, between changing sampling laws. It need not imply small total variation distance.

Let $L_n$ be the probability density function of the absolutely continuous part of $Q_n$ relative to $P_n$. If $L_n$ converges in distribution under $P_n$ to $L>0$ almost surely with $\mathbb{E}L=1$, then $P_n,Q_n$ are mutually contiguous. Indeed, $\mathbb{E}{L}_n\le 1$ and convergence to a mean-one limit imply uniform integrability and vanishing singular mass, giving forward contiguity. For reverse contiguity, $P_n(A_n)\le P_n(L_n\le c)+Q_n(A_n)/c$, and one first takes $n\to \infty$ and then $c\downarrow 0$.

Two sequences of probability laws are mutually contiguous when contiguity of probability measures holds in both directions. A positive mean-one limit of their likelihood ratio is a standard sufficient criterion.

Random variables converge in $L^p$ when $\mathbb{E}|X_n-X{|}^p\to 0$.

Convergence in L1 means that the expected value of the absolute error tends to zero. It implies convergence in probability and convergence of expected values.

Convergence in $L^2$ means $\mathbb{E}[(X_n-X{)}^2]\to 0$. It implies convergence in probability by Markov inequality.

If square-integrable random variables satisfy $X_n\to X$ almost surely, then $X_n\to X$ in $L^2$ exactly when $\mathbb{E}[X_n^2]\to \mathbb{E}[X^2]$. Boundedness in $L^2$ and almost-sure convergence first give weak convergence in the Hilbert space $L^2$; weak convergence together with convergence of norms gives strong convergence.

Random variables $X_n$ converge in distribution, or converge weakly, to $X$ whenfor every bounded continuous function $f$. For real random variables this is equivalent to convergence of the distribution functions at every continuity point of the limiting distribution function.

For convergence in distribution of random elements in a separable complete metric space, one can construct copies of the sequence and its limit on one probability space which converge almost surely. The copies preserve the individual laws; they need not preserve the original dependence between sequence elements. Suitable more general versions permit a separably supported limit in a metric space.

If all moments of real random variables converge to those of a moment-determinate limit law, then the variables converge to that law in the sense of convergence in distribution. Moment determinacy is essential; the standard normal distribution meets this condition.

Probability measures ${\mu }_n$ on a metric space converge weakly to $\mu$ when $\int fd{\mu }_n\to \int fd\mu$ for every bounded continuous real function $f$.

A scaling limit of random curves is a weak convergence of probability measures after spatial rescaling, usually modulo monotone reparameterization. Tightness specifies the topology and yields subsequential limits; identification of their laws gives a unique continuum model. SLE is a central family of such limiting curve laws.

On càdlàg paths on a compact time interval, convergence in this topology means that increasing continuous bijections of the time interval can be chosen converging uniformly to the identity, while the corresponding time-changed paths converge uniformly to the limit. At a continuous limiting path this reduces to uniform convergence. It is a standard topology for weak limits of accelerated discrete-time chains, whose paths are step functions.

This is the coarsest topology making all displayed bounded-continuous test integrals continuous. On a separable metric space it is metrized by the bounded-Lipschitz metric. This probability-measure topology should be distinguished from the weak topology of a Banach space.

The bounded-Lipschitz metric tests probability measures against a uniformly bounded Lipschitz class. It metrizes weak convergence of probability measures on separable metric spaces. With a complete compatible base metric, it is complete on the probability measures of a Polish space. Replacing the sum norm by the maximum norm changes the metric only by uniform constant factors.

On a Polish space, a family of probability measures is relatively compact for weak convergence exactly when it is tight. For finite measures, one additionally requires uniformly bounded total masses.

A weakly closed uniformly tight family of Borel probability measures on a Polish space is weakly compact. Embed the underlying space into a compact metric closure, take subsequential pushforward limits, and use the closed-set Portmanteau theorem on the common tightness compact sets. The limit retains full mass in the original space; inverse continuity of the probability pushforward embedding theorem then returns the weak limit.

On a compact metric space, every sequence of Borel probability measures has a subsequence with weak convergence of probability measures to a Borel probability measure. To see this, choose a countable uniformly dense subset of continuous functions, extract a diagonal subsequence of their bounded integrals, and extend the resulting positive normalized functional to all continuous functions. The Riesz representation theorem supplies its Borel probability measure. This is useful for constructing an invariant measure from orbit empirical measures.

For random variables supported in one fixed compact interval, convergence of every moment is equivalent to weak convergence. One direction tests monomials; the other follows by uniformly approximating continuous functions by polynomials.

Two probability measures supported on one compact interval are equal if all their moments are equal. Indeed, equality on monomials gives equality on polynomials, the Weierstrass approximation theorem extends it to every continuous function on the interval, and compactly supported continuous functions determine a finite Borel measure.

The Portmanteau theorem gives equivalent formulations of weak convergence, including inequalities on open and closed sets and convergence of expectations for every bounded measurable function whose discontinuity set has zero limiting probability.

If $X_n\stackrel{d}{\to }X$ and a measurable function $g$ is continuous at $X$ almost surely, then $g(X_n)\stackrel{d}{\to }g(X)$. The same principle holds for random elements of metric spaces.

A sequence of random variables converges almost surely when the set of outcomes on which pointwise convergence fails has probability zero.

Random variables $X$ and $Y$ are almost surely equal when $\mathbb{P}(X=Y)=1$. Probability theory conventionally identifies random variables that differ only on a null event.

If $(X_n)$ are independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, thenalmost surely.

If ${\sup }_n\mathbb{E}|X_n{|}^2<\infty$ and the $X_n$ are pairwise uncorrelated random variables, then $n^{-1}{\sum }_{k=1}^n(X_k-\mathbb{E}{X}_k)\to 0$ almost surely. Independence is unnecessary for this version of the strong law of large numbers. A uniformly bounded martingale difference sequence meets its hypotheses, since the differences have mean zero and are orthogonal in $L^2$.

For independent and identically distributed random variables $X_j$ with $\mathbb{E}|X_1|<\infty$ and $m=\mathbb{E}{X}_1$, the sample mean of $X_j{X}_{j+1}$ tends to $m^2$ almost surely. Split into odd and even indexed products; each subsequence consists of independent and identically distributed random variables that are integrable random variables. Apply the strong law of large numbers separately.

If independent centered random variables $(Y_n)$ satisfy ${\sum }_n\mathrm{Var}(Y_n)<\infty$, then ${\sum }_n{Y}_n$ converges almost surely. It follows from Kolmogorov's maximal inequality and a Cauchy-tail argument.

For independent identically distributed centered variables of variance one under standard moment assumptions, the law of the iterated logarithm describes the almost-sure envelope of their partial sums on the scale $\sqrt{2n\log \log n}$.

For standard Brownian motion, the Brownian reflection principle bounds the probability that its maximum up to $a^n$ exceeds $(1+\epsilon )\sqrt{2a^n\log \log (a^n)}$ by $2(n\log a{)}^{-(1+\epsilon {)}^2}$. This is summable for every $a>1$ and $\epsilon >0$. The Borel-Cantelli first lemma controls these geometric times, monotonicity of the normalizing function controls the intervening times, and countable choices $a\downarrow 1$, $\epsilon \downarrow 0$ give the bound. Equality requires a separate lower-bound proof.

For a simple symmetric random walk, geometric blocking and the Borel-Cantelli first lemma yield the upper estimate ${\mathrm{limsup}}_n{S}_n/\sqrt{2n\log \log n}\le 1$ almost surely. Equality requires a separate lower-bound argument.

For a simple symmetric random walk, almost surely

Random vectors $X_n$ converge weakly to $X$ exactly when every linear projection $u\cdot X_n$ converges weakly to $u\cdot X$. The reverse implication follows because these one-dimensional limits give pointwise convergence of the multivariate characteristic functions.

Random variables $X_n$ converge in probability to $X$ whenfor every $\epsilon >0$.

Uniform convergence in probability means that the supremum of the absolute discrepancy on the entire index set tends to zero in probability. It controls evaluation at random indices in that set, unlike pointwise convergence in probability. On a fixed compact index set it is a special case of uniform convergence on compacts in probability.

A sequence of random variables $(X_n)$ is bounded in probability when ${\lim }_{K\to \infty }{\sup }_n\mathbb{P}(|X_n|>K)=0$. A sequence converging in probability to a finite random variable has this property, and its product with a sequence tending to zero in probability also tends to zero in probability.

The notation $Z_n=O_{\mathbb{P}}(a_n)$ means that $Z_n/a_n$ is bounded in probability asymptotically: for every $\epsilon >0$ some $C$ gives ${\mathrm{limsup}}_n\mathbb{P}(|Z_n|>C{a}_n)\le \epsilon$. A tail bound $\mathbb{P}(|Z_n|>z{a}_n)\le C_0/z^2$ is sufficient. In statements involving bandwidths, specify whether the constants must be uniform over bandwidth sequences.

A bound on ${\sup }_x\mathbb{P}(|Z_n(x)|>t)$ does not give the same bound on $\mathbb{P}({\sup }_x|Z_n(x)|>t)$. Even independent standard normal random variables each have uniformly bounded tails while their Gaussian maximum over $N$ indices grows at scale $\sqrt{\log N}$. This distinction matters in nonparametric statistics when a shrinking bandwidth permits many spatial windows.

If $X_n\stackrel{d}{\to }c$ for fixed real $c$, convergence of the distribution functions at $c-\epsilon$ and $c+\epsilon$ shows $\mathbb{P}(|X_n-c|>\epsilon )\to 0$. Thus convergence in distribution to a deterministic limit implies convergence in probability to that limit, even though the implication fails for nonconstant limits on a specified probability space.

For independent and identically distributed random variables with finite expected value, the sample mean converges in probability to that expected value.

A class $\mathcal{G}$ satisfies a uniform law of large numbers when its empirical means converge uniformly to their expectations:

A measurable-function class is pointwise separable if it has a countable subclass such that every member is the pointwise limit of a sequence from that subclass. With an integrable common envelope, dominated convergence theorem makes both empirical averages and population expectations converge along each such sequence. Thus the supremum of the absolute empirical errors equals the supremum over the countable subclass and is measurable. A finite integrable bracketing cover provides an integrable envelope by taking the maximum absolute value of all its endpoints.

A bracket $[\ell ,u]$ contains every measurable function $h$ satisfying $\ell \le h\le u$ pointwise. For a probability measure $P$, its $L^1(P)$ width is $P(u-\ell )$, with integrable endpoints. Finite bracketing at every positive width is a sufficient complexity condition for a uniform law of large numbers. Pointwise inequalities, or inequalities outside a single common null set, allow empirical inequalities to hold simultaneously throughout the class.

A measurable function $M$ is an integrable envelope of a function class for a class $\mathcal{H}$ under a probability law $P$ if $|h(x)|\le M(x)$ for every $h\in \mathcal{H}$ and $PM<\infty$. Such an envelope permits the dominated convergence theorem to control shrinking function brackets.

Suppose an integrable measurable-function class can be covered by finitely many function brackets of every positive $L^1(P)$ width. For an independent sample with common law $P$, the empirical measure satisfies ${\sup }_h|P_{n}h-Ph|\to 0$ on a common probability-one event. For one finite $\epsilon$-cover, the supremum is bounded by $\epsilon$ plus the largest empirical error among its endpoints. The strong law of large numbers makes that finite maximum vanish. Taking a countable sequence of widths decreasing to zero proves the assertion. A measurable supremum can be obtained from a pointwise separable function class; otherwise the probability-one-event formulation expresses the same pathwise conclusion.

If $\Theta$ is compact, $q(\theta ,x)$ is continuous in $\theta$ and measurable in $x$, and $\mathbb{E}{\sup }_{\Theta }|q(\theta ,X)|<\infty$, then an independent and identically distributed sample satisfies ${\sup }_{\Theta }|P_n{q}_{\theta }-P{q}_{\theta }|\to 0$ almost surely. Uniform continuity for each $x$, the dominated convergence theorem, and finite parameter nets produce arbitrarily narrow function brackets.

The empirical distribution function converges uniformly almost surely to the population distribution function.

If $X_n$ converges in probability to both $X$ and $Y$, then $X=Y$ almost surely. Indeed, the triangle inequality and a union bound show that $\mathbb{P}(|X-Y|>\epsilon )=0$ for every $\epsilon >0$.

Convergence in probability is equivalent toThe forward implication splits at a fixed error threshold; the reverse implication is Markov's inequality.

From $X_n\to X$ in probability, choose $n_k$ with
$\mathbb{P}(|X_{n_k}-X|>2^{-k})<2^{-k}$. The first Borel--Cantelli lemma then gives $X_{n_k}\to X$ almost surely.

Independent indicators with success probabilities $1/n$ converge to zero in probability, but successes occur infinitely often almost surely by the second Borel--Cantelli lemma.

If $X_n\to X$ in probability and ${\sup }_n\Vert X_n{\Vert }_r<\infty$, then $X_n\to X$ in $L^p$ for every $1\le p<r$. The $L^r$ bound gives uniform integrability of the $p$th powers, while convergence in probability controls their bounded part.

On $([0,1],\mathcal{B},\mathrm{Leb})$, the variablesconverge in probability and in $L^1$ to zero, while $\Vert X_n{\Vert }_2=1$ for every $n$.

A family of integrable random variables is uniformly integrable whenAlmost-sure or probabilistic convergence together with uniform integrability upgrades to $L^1$ convergence.

For $p>1$, ${\sup }_n\mathbb{E}|X_n{|}^p=C<\infty$ gives ${\sup }_n\mathbb{E}[|X_n|1_{|X_n|>M}]\le C{M}^{1-p}\to 0$. Thus an $L^p$ bound implies uniform integrability; a mere $L^1$ bound does not.

For integrable random variables $Z,Y$,Apply this with $Z=X_n$ and $Y=X$ to prove that convergence in L1 implies uniform integrability; control finitely many early terms separately.

For one integrable random variable $Y$, the family $Z_{\mathcal{G}}=\mathbb{E}[Y\mid \mathcal{G}]$, over arbitrary sigma-algebras $\mathcal{G}$, has uniform integrability. On $A=\{|Z_{\mathcal{G}}|>K\}$, the defining property of conditional expectation gives $\mathbb{E}[|Z_{\mathcal{G}}|1_A]\le \mathbb{E}[|Y|1_A]$, while $\mathbb{P}(A)\le \mathbb{E}|Y|/K$ by the Markov inequality. The integrable random variable $Y$ has uniformly small absolute integrals over events of sufficiently small probability.

If ${\sup }_n\mathbb{E}[X_n^2]<\infty$, then $(X_n)$ is uniformly integrable. Indeed, the Cauchy-Schwarz inequality and Markov inequality give

A sequence $(Z_n)$ satisfies a large deviation principle with speed $n$ and rate function $I$ when probabilities of measurable sets $A$ decay exponentially between the bounds

A rate function is a lower-semicontinuous map $I$ into $[0,\infty ]$. It is good when every sublevel set $\{x:I(x)\le c\}$ is compact.

For a small-noise diffusion $dX=a(X)dt+\sqrt{ϵ}\sigma (X)dW$, the Freidlin--Wentzell action of a smooth path is

On a zero-Hamiltonian infinite-time fluctuation path, the Freidlin--Wentzell action can be parametrized independently of time as $I=\int \Vert a\Vert ds-a\cdot d\varphi$ in the diffusion metric.

A normal approximation replaces a sampling distribution by a normal distribution, often through a central limit theorem. In a Wald test, approximate statistical power is determined by the standardized difference under the alternative. This yields Neyman allocation for fixed treatment-specific variances.

The Lindeberg condition requires the contribution of unusually large summands in a triangular array to become negligible compared with the array's total variance. Together with convergence of that variance, it gives the Lindeberg central limit theorem.

For independent identically distributed centered random variables in a separable Hilbert space with finite second moment, $n^{-1/2}{\sum }_{i=1}^n{X}_i$ converges in distribution to a centered Gaussian random element with the same covariance operator.

For independent identically distributed $Z_i$ with mean zero and variance one,The numerator obeys the central limit theorem, the denominator converges almost surely to one by the strong law, and Slutsky's theorem combines them.

For independent identically distributed increments of mean zero and variance one, the diffusively rescaled, linearly interpolated random walk converges weakly in $C[0,1]$ to Brownian motion.

For a random walk with centered independent identically distributed increments of variance one, the maximum of its polygonal interpolation is the maximum at its grid vertices. The maximum functional is Lipschitz for the supremum norm, so the continuous mapping theorem applied to the Donsker invariance principle proves the displayed convergence in distribution. The Brownian reflection principle then gives upper-tail limits $2(1-\Phi (a))$ at all $a>0$, since the limiting maximum has no atom there.

For a random walk with independent identically distributed steps of mean zero and variance one, the Donsker invariance principle and the continuous mapping theorem for integration give this limit. The integral of the polygonal interpolation differs from the right-endpoint sum by $S_n/(2n^{3/2})$, whose squared expectation is $1/(4n^2)$. The Slutsky theorem removes that error. The limiting variable is the time-one value of integrated Brownian motion, with law $N(0,1/3)$.

For independent identically distributed random vectors $Z_i$ with mean $\mu$ and finite covariance matrix $V$,

Binomial distributions with trial count tending to infinity and success probability tending to zero converge to a Poisson law when their product converges.