Convergence of random variables

The main modes include almost sure convergence, convergence in probability, convergence in Lp, and convergence in distribution. The strong law of large numbers and weak law of large numbers explain why a sample mean approaches an expected value; the central limit theorem describes the probability distribution of its remaining error after magnification by $\sqrt{n}$.

A sequence $Q_n$ is contiguous with respect to $P_n$ if $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$ for every measurable sequence of events. It transfers negligible-event assertions, including consistency, between changing sampling laws. It need not imply small total variation distance.

Let $L_n$ be the probability density function of the absolutely continuous part of $Q_n$ relative to $P_n$. If $L_n$ converges in distribution under $P_n$ to $L>0$ almost surely with $\mathbb{E}L=1$, then $P_n,Q_n$ are mutually contiguous. Indeed, $\mathbb{E}{L}_n\le 1$ and convergence to a mean-one limit imply uniform integrability and vanishing singular mass, giving forward contiguity. For reverse contiguity, $P_n(A_n)\le P_n(L_n\le c)+Q_n(A_n)/c$, and one first takes $n\to \infty$ and then $c\downarrow 0$.

Two sequences of probability laws are mutually contiguous when contiguity of probability measures holds in both directions. A positive mean-one limit of their likelihood ratio is a standard sufficient criterion.

Random variables converge in $L^p$ when $\mathbb{E}|X_n-X{|}^p\to 0$.

Convergence in L1 means that the expected value of the absolute error tends to zero. It implies convergence in probability and convergence of expected values.

For nonnegative integrable functions converging almost everywhere to an integrable $f$, convergence of their integrals implies L1 convergence. Indeed $\min (f_n,f)\to f$ almost everywhere and is dominated by $f$, so the dominated convergence theorem gives convergence of its integral. Use $|f_n-f|=f_n+f-2\min (f_n,f)$. This applies to probability densities, giving convergence in total variation distance.

Convergence in $L^2$ means $\mathbb{E}[(X_n-X{)}^2]\to 0$. It implies convergence in probability by Markov inequality.

If square-integrable random variables satisfy $X_n\to X$ almost surely, then $X_n\to X$ in $L^2$ exactly when $\mathbb{E}[X_n^2]\to \mathbb{E}[X^2]$. Boundedness in $L^2$ and almost-sure convergence first give weak convergence in the Hilbert space $L^2$; weak convergence together with convergence of norms gives strong convergence.

Random variables $X_n$ converge in distribution, or converge weakly, to $X$ whenfor every bounded continuous function $f$. For real random variables this is equivalent to convergence of the distribution functions at every continuity point of the limiting distribution function.

The probability distribution approached in convergence in distribution. It can depend on the initial state of a Markov chain when several closed communicating classes are accessible.

If $Z_n$ has probability at least p at $z_n$, with $p>0$ and $z_n\to z$, every weak limit has probability at least p at z. Apply the closed-set bound of the Portmanteau theorem to arbitrarily small closed intervals containing z and eventually $z_n$, then decrease their radii. Thus a persistent atom prevents convergence to a continuous law, including a chi-squared law. This provides a direct check when bounded-information estimation fails standard asymptotic-normal theory.

Let $Z$ take values $1,-1$ with equal probabilities. Set $X_n=Z$ for every $n$, and $Y_n=Z$ for even $n$, $Y_n=-Z$ for odd $n$. Both marginal laws are constant, but $X_n+Y_n$ alternates between the law of $2Z$ and a point mass at zero. Thus separate weak convergence of random variables does not determine a joint limit or a sum limit. The obstruction disappears when one marginal limit is constant, as in the Slutsky theorem.

For convergence in distribution of random elements in a separable complete metric space, one can construct copies of the sequence and its limit on one probability space which converge almost surely. The copies preserve the individual laws; they need not preserve the original dependence between sequence elements. Suitable more general versions permit a separably supported limit in a metric space.

If all moments of real random variables converge to those of a moment-determinate limit law, then the variables converge to that law in the sense of convergence in distribution. Moment determinacy is essential; the standard normal distribution meets this condition.

Probability measures ${\mu }_n$ on a metric space converge weakly to $\mu$ when $\int fd{\mu }_n\to \int fd\mu$ for every bounded continuous real function $f$.

A family of path laws satisfies compact containment when for every finite horizon and every error tolerance there is a compact set which contains the whole path up to that horizon with at least the complementary probability, uniformly over the family. In Euclidean state space this is the displayed condition. It prevents escape to distant regions where local coefficient approximations give no control. A nonnegative Lyapunov function with uniformly bounded generator growth often proves it by the optional stopping theorem.

A scaling limit of random curves is a weak convergence of probability measures after spatial rescaling, usually modulo monotone reparameterization. Tightness specifies the topology and yields subsequential limits; identification of their laws gives a unique continuum model. SLE is a central family of such limiting curve laws.

On càdlàg paths on a compact time interval, convergence in this topology means that increasing continuous bijections of the time interval can be chosen converging uniformly to the identity, while the corresponding time-changed paths converge uniformly to the limit. At a continuous limiting path this reduces to uniform convergence. It is a standard topology for weak limits of accelerated discrete-time chains, whose paths are step functions.

This is the coarsest topology making all displayed bounded-continuous test integrals continuous. On a separable metric space it is metrized by the bounded-Lipschitz metric. This probability-measure topology should be distinguished from the weak topology of a Banach space.

The bounded-Lipschitz metric tests probability measures against a uniformly bounded Lipschitz class. It metrizes weak convergence of probability measures on separable metric spaces. With a complete compatible base metric, it is complete on the probability measures of a Polish space. Replacing the sum norm by the maximum norm changes the metric only by uniform constant factors.

On a Polish space, a family of probability measures is relatively compact for weak convergence exactly when it is tight. For finite measures, one additionally requires uniformly bounded total masses.

A weakly closed uniformly tight family of Borel probability measures on a Polish space is weakly compact. Embed the underlying space into a compact metric closure, take subsequential pushforward limits, and use the closed-set Portmanteau theorem on the common tightness compact sets. The limit retains full mass in the original space; inverse continuity of the probability pushforward embedding theorem then returns the weak limit.

On a compact metric space, every sequence of Borel probability measures has a subsequence with weak convergence of probability measures to a Borel probability measure. To see this, choose a countable uniformly dense subset of continuous functions, extract a diagonal subsequence of their bounded integrals, and extend the resulting positive normalized functional to all continuous functions. The Riesz representation theorem supplies its Borel probability measure. This is useful for constructing an invariant measure from orbit empirical measures.

For random variables supported in one fixed compact interval, convergence of every moment is equivalent to weak convergence. One direction tests monomials; the other follows by uniformly approximating continuous functions by polynomials.

Two probability measures supported on one compact interval are equal if all their moments are equal. Indeed, equality on monomials gives equality on polynomials, the Weierstrass approximation theorem extends it to every continuous function on the interval, and compactly supported continuous functions determine a finite Borel measure.

The Portmanteau theorem gives equivalent formulations of weak convergence, including inequalities on open and closed sets and convergence of expectations for every bounded measurable function whose discontinuity set has zero limiting probability.

If $X_n\stackrel{d}{\to }X$ and a measurable function $g$ is continuous at $X$ almost surely, then $g(X_n)\stackrel{d}{\to }g(X)$. The same principle holds for random elements of metric spaces.

Laws of large numbers give conditions under which sample means approach a deterministic expected value. For independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, the strong law of large numbers gives almost sure convergence, while the weak law of large numbers gives convergence in probability. These conclusions specify different modes of convergence of random variables, rather than a universal numerical accuracy guarantee. The central limit theorem is a separate result about the remaining fluctuations after rescaling.

A sequence of random variables converges almost surely when the set of outcomes on which pointwise convergence fails has probability zero.

A sequence converges completely to $X$ when ${\sum }_n\mathrm{Pr}(|X_n-X|>ϵ)<\infty$ for every $ϵ>0$. The first Borel-Cantelli lemma implies almost sure convergence. For independent $X_n$, any limit in probability is almost surely constant; the second Borel-Cantelli lemma then gives the converse from almost sure to complete convergence. Independence is essential: $X_n=1_{\{U\le 1/n\}}$ for one uniform random variable $U$ converges almost surely to zero but its exceedance probabilities have a divergent sum.

If independent random variables $X_n$ converge almost surely to a finite $X$, then ${\mathrm{limsup}}_n{X}_n$ is measurable for the tail sigma-algebra. The Kolmogorov zero-one law forces every rational threshold event for that limit to have probability zero or one. A real-valued probability distribution with this property is a point mass, so $X$ is almost surely constant. Consequently the Second Borel-Cantelli lemma upgrades almost sure convergence of independent sequences to complete convergence.

Random variables $X$ and $Y$ are almost surely equal when $\mathbb{P}(X=Y)=1$. Probability theory conventionally identifies random variables that differ only on a null event.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, and let $\mu =\mathbb{E}[X_1]$ be their expected value. Their sample mean satisfiesThis is almost sure convergence: with probability one, an entire infinite sequence of observations has averages that eventually stay within every prescribed positive tolerance of $\mu$. The variance need not be finite. The strong law of large numbers implies the weak law of large numbers, which asserts convergence in probability at each increasing sample size.

For independent trials with a Bernoulli distribution of parameter $p$, $X_j$ is one for success and zero for failure. Then ${\overline{X}}_n$ is the observed success proportion, and the strong law of large numbers says ${\overline{X}}_n\to p$ almost surely. This turns a theoretical probability into a quantity one can estimate by repetition. It does not say that each new trial improves the estimate: a long string of failures can move the proportion away from $p$ before later observations bring it back.

The strong law of large numbers gives the limiting location of the sample mean, while the central limit theorem describes its fluctuations when the variance is finite and positive. The two assertions can hold together:The first error shrinks to zero; the second has been enlarged enough to retain a nontrivial probability distribution. The second assertion is convergence in distribution, not a claim that the enlarged error along one sequence approaches a particular normal random variable.

The assumptions make a practical difference. If every observation repeats the same nonconstant Bernoulli random variable, $X_j=X_1$, then ${\overline{X}}_n=X_1$ for all $n$: identical probability distributions alone do not make repeated observations informative. For independent observations with a Standard Cauchy distribution, the Cauchy sample mean has an unchanged distribution, so it does not concentrate around a constant. Its expected value is undefined, and $\mathbb{E}|X_1|=\infty$; neither example contradicts the stated strong law of large numbers.

A brief proof outline in the case of finite variance uses the Kolmogorov maximal inequality for the centered partial sums. The probability that their maximum up to $2^m$ exceeds $\epsilon 2^m$ is at most ${\sigma }^2/({\epsilon }^2{2}^m)$. These bounds are summable, so the Borel-Cantelli lemma controls all sufficiently large such blocks and hence every intermediate sample size. The general case $\mathbb{E}|X_1|<\infty$ requires truncation to control large observations; the argument using finite variance alone does not prove that full version.

For independent identically distributed random variables, the tail-sum formula applied to $|X_1|/\lambda$ shows that ${\sum }_n\mathbb{P}(|X_n|\ge \lambda n)=\infty$ for every $\lambda >0$ when the first absolute moment is infinite. The Borel-Cantelli second lemma, followed by a countable intersection over positive integer $\lambda$, gives the first limsup assertion.

If $|S_n|/n$ had finite limsup along such a sample path, it would eventually be bounded by some finite constant $C$. Then $X_n=S_n-S_{n-1}$ would imply $|X_n|/n\le 2C$ eventually, a contradiction. This is a necessary-integrability counterpart to the strong law of large numbers; without a finite first absolute moment, the normalized sums cannot converge to any finite value.

Take independent random variables equal to $1+n$ and $1-n$ with probability $1/(2n\log n)$ each, and otherwise equal to one. Their first absolute deviations are $1/\log n\to 0$, so $X_n\to 1$ in L1 convergence. The same triangle-inequality argument gives convergence in $L^1$ of their arithmetic averages to one. For $p>1$, however, $\mathbb{E}|X_n-1{|}^p=n^{p-1}/\log n\to \infty$.

The exceptional jumps occur infinitely often almost surely by the Borel-Cantelli second lemma, since $\sum 1/(n\log n)=\infty$. If the arithmetic averages $S_n/n$ converged to any finite value along a sample path, then $X_{n+1}/n=S_n/n-[(n-1)/n][S_{n-1}/(n-1)]$ would tend to zero. At each exceptional jump its magnitude is at least one. Thus the averages have no finite almost sure limit. Independence and convergence of first-moment averages do not alone give a strong law of large numbers.

If the positive factors $U_j$ are independent and identically distributed and $\mathbb{E}|\log U_1|<\infty$, the strong law of large numbers applied to their logarithms gives the displayed almost sure convergence. A negative limit forces the products to tend to zero; a positive limit forces them to tend to infinity. A zero limit alone gives neither conclusion. For a uniform distribution on $(0,b)$, the logarithmic expected value is $\log b-1$, so the products tend to zero almost surely for $b<e$ and to infinity almost surely for $b>e$.

If ${\sup }_n\mathbb{E}|X_n{|}^2<\infty$ and the $X_n$ are pairwise uncorrelated random variables, then $n^{-1}{\sum }_{k=1}^n(X_k-\mathbb{E}{X}_k)\to 0$ almost surely. Independence is unnecessary for this version of the strong law of large numbers. A uniformly bounded martingale difference sequence meets its hypotheses, since the differences have mean zero and are orthogonal in $L^2$.

For independent and identically distributed random variables $X_j$ with $\mathbb{E}|X_1|<\infty$ and $m=\mathbb{E}{X}_1$, the sample mean of $X_j{X}_{j+1}$ tends to $m^2$ almost surely. Split into odd and even indexed products; each subsequence consists of independent and identically distributed random variables that are integrable random variables. Apply the strong law of large numbers separately.

If independent centered random variables $(Y_n)$ satisfy ${\sum }_n\mathrm{Var}(Y_n)<\infty$, then ${\sum }_n{Y}_n$ converges almost surely. It follows from Kolmogorov's maximal inequality and a Cauchy-tail argument.

For independent identically distributed centered variables of variance one under standard moment assumptions, the law of the iterated logarithm describes the almost-sure envelope of their partial sums on the scale $\sqrt{2n\log \log n}$.

For standard Brownian motion, the Brownian reflection principle bounds the probability that its maximum up to $a^n$ exceeds $(1+\epsilon )\sqrt{2a^n\log \log (a^n)}$ by $2(n\log a{)}^{-(1+\epsilon {)}^2}$. This is summable for every $a>1$ and $\epsilon >0$. The Borel-Cantelli first lemma controls these geometric times, monotonicity of the normalizing function controls the intervening times, and countable choices $a\downarrow 1$, $\epsilon \downarrow 0$ give the bound. Equality requires a separate lower-bound proof.

For a simple symmetric random walk, geometric blocking and the Borel-Cantelli first lemma yield the upper estimate ${\mathrm{limsup}}_n{S}_n/\sqrt{2n\log \log n}\le 1$ almost surely. Equality requires a separate lower-bound argument.

For a simple symmetric random walk, almost surely

Random vectors $X_n$ converge weakly to $X$ exactly when every linear projection $u\cdot X_n$ converges weakly to $u\cdot X$. The reverse implication follows because these one-dimensional limits give pointwise convergence of the multivariate characteristic functions.

Random variables $X_n$ converge in probability to $X$ whenfor every $\epsilon >0$.

For real random variables on one probability space, fix a continuity point $x$ of the cumulative distribution function $F_X$. For every $\epsilon >0$, inclusions of events giveTake the lower and upper limits as $n\to \infty$, using convergence in probability, and then let $\epsilon \downarrow 0$. Continuity at $x$ makes both bounds $F_X(x)$, proving convergence in distribution. The two modes of convergence have different information: convergence in probability concerns the coupling on one probability space, whereas convergence in distribution concerns only the marginal probability distributions.

For $\alpha >0$, Markov's inequality gives the displayed bound, so $\mathbb{E}|Y_n{|}^{\alpha }\to 0$ implies convergence in probability to zero even when the first absolute moment does not tend to zero. For products $Y_n={\prod }_{j=1}^n{U}_j$ of independent and identically distributed factors, it suffices that $\mathbb{E}|U_1{|}^{\alpha }<1$. With the uniform distribution on $(0,b)$, the product of independent uniform random variables formula makes this $\alpha \log b<\log (1+\alpha )$. Since $\log (1+\alpha )/\alpha \to 1$ as $\alpha \downarrow 0$, such an $\alpha$ exists whenever $b<e$.

Uniform convergence in probability means that the supremum of the absolute discrepancy on the entire index set tends to zero in probability. It controls evaluation at random indices in that set, unlike pointwise convergence in probability. On a fixed compact index set it is a special case of uniform convergence on compacts in probability.

A sequence of random variables $(X_n)$ is bounded in probability when ${\lim }_{K\to \infty }{\sup }_n\mathbb{P}(|X_n|>K)=0$. A sequence converging in probability to a finite random variable has this property, and its product with a sequence tending to zero in probability also tends to zero in probability.

The notation $Z_n=O_{\mathbb{P}}(a_n)$ means that $Z_n/a_n$ is bounded in probability asymptotically: for every $\epsilon >0$ some $C$ gives ${\mathrm{limsup}}_n\mathbb{P}(|Z_n|>C{a}_n)\le \epsilon$. A tail bound $\mathbb{P}(|Z_n|>z{a}_n)\le C_0/z^2$ is sufficient. In statements involving bandwidths, specify whether the constants must be uniform over bandwidth sequences.

A bound on ${\sup }_x\mathbb{P}(|Z_n(x)|>t)$ does not give the same bound on $\mathbb{P}({\sup }_x|Z_n(x)|>t)$. Even independent standard normal random variables each have uniformly bounded tails while their Gaussian maximum over $N$ indices grows at scale $\sqrt{\log N}$. This distinction matters in nonparametric statistics when a shrinking bandwidth permits many spatial windows.

If $X_n\stackrel{d}{\to }c$ for fixed real $c$, convergence of the distribution functions at $c-\epsilon$ and $c+\epsilon$ shows $\mathbb{P}(|X_n-c|>\epsilon )\to 0$. Thus convergence in distribution to a deterministic limit implies convergence in probability to that limit, even though the implication fails for nonconstant limits on a specified probability space.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$ and expected value $\mu =\mathbb{E}[X_1]$. The weak law of large numbers states that their sample mean has convergence in probability to $\mu$: for every fixed $\epsilon >0$,Thus a large sample is unlikely to give an average with a specified fixed-size error. The strong law of large numbers makes the stronger statement of almost sure convergence along an entire sequence. Under the displayed assumptions both laws apply; their names distinguish the conclusions, not two competing predictions.

When the variance ${\sigma }^2$ is finite, variance additivity for independent random variables gives $\mathrm{Var}({\overline{X}}_n)={\sigma }^2/n$. The Chebyshev inequality immediately supplies a useful quantitative form and a brief proof:For independent trials with a Bernoulli distribution of parameter $p=0.2$, the expected value is $0.2$ and the variance is $0.16$. At $n=1000$, the chance of an error of at least $0.1$ is therefore at most $0.016$. The exact binomial distribution gives a much smaller probability, as the figure under the strong law of large numbers shows. The bound is conservative, but it is valid for every sample size under these assumptions and does not require a normal approximation.

The weak law of large numbers addresses a fixed tolerance $\epsilon$. The central limit theorem instead resolves errors on the shrinking scale $\sigma /\sqrt{n}$ and gives their approximate normal distribution. In particular, the weak law of large numbers alone supplies neither a bell-shaped error law nor a universal rate of approach. For example, a Pareto distribution with minimum one and shape $3/2$ has expected value three but infinite variance: both the weak law of large numbers and strong law of large numbers apply, while the finite-variance statement of the central limit theorem does not.

Suppose the variances of a sequence of random variables are bounded by $C$ and $\mathrm{Cov}(X_i,X_j)=0$ whenever $|i-j|>r$, for a fixed nonnegative integer $r$. The Cauchy-Schwarz inequality bounds each remaining covariance in absolute value by $C$. In the variance of a sum, there are at most $rn$ off-diagonal pairs, proving the displayed bound. The Chebyshev inequality makes the average minus its mean converge to zero in probability. If all means equal $\mu$, the average therefore converges in probability to $\mu$. Independence of every summand is unnecessary.

Let $X$ have a symmetric probability distribution with $\mathbb{P}(|X|>x)=1/(x\log x)$ for $x\ge e$, mass $1-1/e$ at zero, and no mass with $0<|X|<e$. Then $\mathbb{E}|X|=\infty$, but independent copies satisfy the weak law of large numbers with limit zero. Indeed $n\mathbb{P}(|X|>n)=1/\log n\to 0$. Their symmetric truncations $Y_{n,j}=X_j{1}_{\{|X_j|\le n\}}$ have mean zero and second moment $O(n/\log n)$, so Chebyshev's inequality gives $n^{-1}{\sum }_{j\le n}{Y}_{n,j}\to 0$ in probability. A union bound shows that the original and truncated sums differ with probability tending to zero. This also gives a differentiable characteristic function at zero whose derivative cannot be interpreted as an absolutely convergent expectation.

For independent and identically distributed random variables with characteristic function $\varphi$, suppose their sample mean converges in probability to a finite constant $a$. Bounded exponential tests give $e^{-iau}\varphi (u/n{)}^n\to 1$ uniformly on bounded $u$ intervals: the error is at most $T\delta +2\mathbb{P}(|S_n/n-a|>\delta )$ on $|u|\le T$. Taking the continuous complex logarithm vanishing at zero shows $n\log \varphi (u/n)-iau\to 0$ uniformly; an integer-valued branch difference is continuous and therefore zero. For arbitrary $h\to 0$, put $n=\lfloor 1/|h|\rfloor$ and $u=nh$, so $1/2\le |u|\le 1$. Dividing the uniform limit by $u$ proves $\log \varphi (h)/h\to ia$, and thus ${\varphi }^{\prime }(0)=ia$. No absolute first moment is assumed.

A class $\mathcal{G}$ satisfies a uniform law of large numbers when its empirical means converge uniformly to their expectations:

A measurable-function class is pointwise separable if it has a countable subclass such that every member is the pointwise limit of a sequence from that subclass. With an integrable common envelope, dominated convergence theorem makes both empirical averages and population expectations converge along each such sequence. Thus the supremum of the absolute empirical errors equals the supremum over the countable subclass and is measurable. A finite integrable bracketing cover provides an integrable envelope by taking the maximum absolute value of all its endpoints.

A bracket $[\ell ,u]$ contains every measurable function $h$ satisfying $\ell \le h\le u$ pointwise. For a probability measure $P$, its $L^1(P)$ width is $P(u-\ell )$, with integrable endpoints. Finite bracketing at every positive width is a sufficient complexity condition for a uniform law of large numbers. Pointwise inequalities, or inequalities outside a single common null set, allow empirical inequalities to hold simultaneously throughout the class.

A measurable function $M$ is an integrable envelope of a function class for a class $\mathcal{H}$ under a probability law $P$ if $|h(x)|\le M(x)$ for every $h\in \mathcal{H}$ and $PM<\infty$. Such an envelope permits the dominated convergence theorem to control shrinking function brackets.

Suppose an integrable measurable-function class can be covered by finitely many function brackets of every positive $L^1(P)$ width. For an independent sample with common law $P$, the empirical measure satisfies ${\sup }_h|P_{n}h-Ph|\to 0$ on a common probability-one event. For one finite $\epsilon$-cover, the supremum is bounded by $\epsilon$ plus the largest empirical error among its endpoints. The strong law of large numbers makes that finite maximum vanish. Taking a countable sequence of widths decreasing to zero proves the assertion. A measurable supremum can be obtained from a pointwise separable function class; otherwise the probability-one-event formulation expresses the same pathwise conclusion.

If $\Theta$ is compact, $q(\theta ,x)$ is continuous in $\theta$ and measurable in $x$, and $\mathbb{E}{\sup }_{\Theta }|q(\theta ,X)|<\infty$, then an independent and identically distributed sample satisfies ${\sup }_{\Theta }|P_n{q}_{\theta }-P{q}_{\theta }|\to 0$ almost surely. Uniform continuity for each $x$, the dominated convergence theorem, and finite parameter nets produce arbitrarily narrow function brackets.

The empirical distribution function converges uniformly almost surely to the population distribution function.

For a cumulative distribution function $G$, let $Q(u)=\inf \{x:G(x)\ge u\}$, $0<u<1$. Right continuity gives $Q(u)\le x$ exactly when $u\le G(x)$, including at atoms. Thus applying $Q$ to independent uniform variables $U_i$ constructs an independent sample with distribution function $G$. Its empirical distribution function is $G_n(x)=F_n(G(x))$, where $F_n$ is the uniform-sample empirical distribution function. Consequently ${\sup }_x|G_n(x)-G(x)|\le {\sup }_{t\in [0,1]}|F_n(t)-t|$. For the uniform sample, monotonicity and a grid of mesh $1/m$ bound the last supremum by the largest error at grid points plus $1/m$. The strong law of large numbers on the countable union of all such grids proves the almost-sure limit. This construction proves the theorem for distributions with atoms as well as continuous distributions.

If $X_n$ converges in probability to both $X$ and $Y$, then $X=Y$ almost surely. Indeed, the triangle inequality and a union bound show that $\mathbb{P}(|X-Y|>\epsilon )=0$ for every $\epsilon >0$.

Convergence in probability is equivalent toThe forward implication splits at a fixed error threshold; the reverse implication is Markov's inequality.

From $X_n\to X$ in probability, choose $n_k$ with
$\mathbb{P}(|X_{n_k}-X|>2^{-k})<2^{-k}$. The first Borel--Cantelli lemma then gives $X_{n_k}\to X$ almost surely.

Independent indicators with success probabilities $1/n$ converge to zero in probability, but successes occur infinitely often almost surely by the second Borel--Cantelli lemma.

If $X_n\to X$ in probability and ${\sup }_n\Vert X_n{\Vert }_r<\infty$, then $X_n\to X$ in $L^p$ for every $1\le p<r$. The $L^r$ bound gives uniform integrability of the $p$th powers, while convergence in probability controls their bounded part.

On $([0,1],\mathcal{B},\mathrm{Leb})$, the variablesconverge in probability and in $L^1$ to zero, while $\Vert X_n{\Vert }_2=1$ for every $n$.

A family of integrable random variables is uniformly integrable whenAlmost-sure or probabilistic convergence together with uniform integrability upgrades to $L^1$ convergence.

On a finite measure space, a family is uniformly integrable exactly when it admits the displayed bound for some nonnegative increasing convex superlinear function $\Psi$. By the Dunford-Pettis theorem this ensures relative weak compactness in $L^1$. For the sufficient direction, choose $R_A$ so $\Psi (s)\ge As$ for $s>R_A$; then ${\int }_E|v_j|\le R_A\mu (E)+A^{-1}{\sup }_j\int \Psi (|v_j|)$ gives uniform integrability. Conversely choose increasing thresholds $R_m\to \infty$ so ${\sup }_j{\int }_{|v_j|>R_m}|v_j|\le 2^{-m}$. The convex function $\Psi (s)={\sum }_{m\ge 1}(s-R_m{)}_{+}$ is locally a finite sum, has uniformly bounded integrals by summing these tail estimates, and is superlinear because for $s\ge 2R_N$ its first $N$ terms sum to at least $Ns/2$. This proves the criterion in both directions.

Choose increasing $R_m\to \infty$ so $H(t)\ge m^2|t|$ whenever $|t|\ge R_m$. Put $\Psi (s)={\sum }_{m\ge 1}(s-R_m{)}_{+}$ for $s\ge 0$. This is nonnegative, increasing and convex. If $m$ terms are active, then $\Psi (s)\le ms\le H(t)$ for $s=|t|$. For every fixed $N$, $s\ge 2R_N$ gives $\Psi (s)\ge Ns/2$, proving superlinearity. Thus a nonconvex superlinear energy lower bound still supplies the convex function needed in the De la Vallée-Poussin criterion. A continuous superlinear function not initially nonnegative can first have its finite global minimum subtracted.

On a finite-measure space, a bounded subset of $L^1$ is relatively weakly compact exactly when it is uniformly integrable. In particular every uniformly integrable bounded sequence has a weakly convergent subsequence. This is the compactness substitute for reflexivity at exponent one. A superlinear integral bound ${\sup }_j\int H(u_j)<\infty$, with $H\ge 0$ and $H(t)/|t|\to \infty$, gives the required uniform integrability: for $H(t)\ge A|t|$ outside $[-R_A,R_A]$,Choose $A$ large and then the measure of $E$ small. This is the direction used in the direct method in the calculus of variations; a mere bounded $L^1$ norm does not suffice, as concentrating spikes show.

For a uniformly integrable bounded sequence of derivatives on a compact interval and a fixed initial value, its primitives are uniformly bounded and equicontinuous. The increment over any short interval is bounded by the integral of the absolute derivative on that interval. The Arzelà-Ascoli theorem gives relative uniform compactness. If the derivatives converge weakly in $L^1$, tests by interval indicators identify every pointwise limit with $u(x)=\alpha +{\int }_a^{x}v$. Hence every convergent subsequence has this same limit and the primitives converge uniformly. The limit belongs to $W^{1,1}$ and retains any common terminal value. Uniform integrability alone gives no essential-supremum bound on its derivative, so this compactness statement does not assert a Lipschitz limit.

On a finite time horizon, an adapted process is of class D if its values at all stopping times in that horizon form a uniformly integrable family. Deterministic boundedness implies this property. The stopping-time uniformity is stronger than having an integrable value at each fixed time.

An adapted process is of class DL when, for every finite $T$, the family $\{X_{\tau }:\tau \le T\text{ a stopping time}\}$ is uniformly integrable. A local martingale is a true martingale exactly when it has this property. This is a local-in-time condition, not uniform integrability over the whole half-line.

// Target: probability-and-statistics.bigb

For $p>1$, ${\sup }_n\mathbb{E}|X_n{|}^p=C<\infty$ gives ${\sup }_n\mathbb{E}[|X_n|1_{|X_n|>M}]\le C{M}^{1-p}\to 0$. Thus an $L^p$ bound implies uniform integrability; a mere $L^1$ bound does not.

For integrable random variables $Z,Y$,Apply this with $Z=X_n$ and $Y=X$ to prove that convergence in L1 implies uniform integrability; control finitely many early terms separately.

For one integrable random variable $Y$, the family $Z_{\mathcal{G}}=\mathbb{E}[Y\mid \mathcal{G}]$, over arbitrary sigma-algebras $\mathcal{G}$, has uniform integrability. On $A=\{|Z_{\mathcal{G}}|>K\}$, the defining property of conditional expectation gives $\mathbb{E}[|Z_{\mathcal{G}}|1_A]\le \mathbb{E}[|Y|1_A]$, while $\mathbb{P}(A)\le \mathbb{E}|Y|/K$ by the Markov inequality. The integrable random variable $Y$ has uniformly small absolute integrals over events of sufficiently small probability.

If ${\sup }_n\mathbb{E}[X_n^2]<\infty$, then $(X_n)$ is uniformly integrable. Indeed, the Cauchy-Schwarz inequality and Markov inequality give

A sequence $(Z_n)$ satisfies a large deviation principle with speed $n$ and rate function $I$ when probabilities of measurable sets $A$ decay exponentially between the bounds

If a large deviation principle has a finite continuous rate function, then the infima on any open set and its closure agree. Approximate every closure point by points of the set and use continuity. The open-set lower and closed-set upper bounds consequently match, giving the displayed logarithmic limit, including the empty-set convention. A single-point event need not obey such an equality.

A large deviation principle for fluctuations larger than the usual central limit theorem scale but smaller than a fixed displacement of the empirical mean. For the displayed sum scaling, the natural large-deviation speed is $b_n^2$. Under suitable exponential-moment hypotheses and positive variance ${\sigma }^2$, its rate function is the quadratic $x^2/(2{\sigma }^2)$.