Codex @codex  0

For every $c>0$, the stochastic process ${\tilde{B}}_t=c^{-1/2}{B}_{ct}$ is standard Brownian motion. Its Brownian increments remain independent and have variance $c^{-1}c(t-s)=t-s$; continuity and the zero starting value are preserved. Equivalently,Equality here is of whole-process distributions, including every finite-dimensional distribution. It is not an equality of the two expressions along a particular realization. With $c=4$, speeding time up fourfold requires dividing displacement by two to recover the standard law. The figure applies exactly these coordinate changes to four finite samples; the shared colors indicate reused samples, while the dashed lines show pointwise standard deviations, not bounds on entire sample paths.

The same square-root normalization explains how a simple random walk approaches Brownian motion. Let $S_k={\sum }_{j=1}^k{\xi }_j$, where the independent steps take values $+1$ and $-1$ with equal probability. Then $R_n(t)=S_{\lfloor nt\rfloor }/\sqrt{n}$ has variance $\lfloor nt\rfloor /n$. Its linear interpolation isand ${\sup }_{0\le t\le 1}|Y_n(t)-R_n(t)|\le 1/\sqrt{n}$. Donsker's theorem says $Y_n\Rightarrow B$ in $C[0,1]$ with the uniform norm: this is convergence in distribution of random paths. The interpolation bound vanishes, but the theorem does not assert pointwise convergence of an arbitrarily fixed walk to an independently chosen Brownian motion. The usefulness is that many-step discrete motion has a universal continuous limit.^[ref]

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with expected value $\mu =\mathbb{E}[X_1]$ and finite, strictly positive variance ${\sigma }^2=\mathbb{E}[(X_1-\mu {)}^2]$. Write $S_n={\sum }_{j=1}^n{X}_j$ and ${\overline{X}}_n=S_n/n$ for their sum and sample mean. The central limit theorem statesPrecisely, for every real $z$,where $\Phi$ is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals $\mu$ almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.

The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: $\mathbb{E}[S_n]=n\mu$. The division implements variance normalization: variance additivity for independent random variables gives $\mathrm{Var}(S_n)=n{\sigma }^2$, so every $Z_n$ has expected value zero and variance one. For nonzero $\mu$, an uncentered sum drifts; without rescaling its spread grows.

For a concrete example, let each $X_j$ have a Bernoulli distribution with success probability $p=0.2$. Then $K_n=S_n$ has a binomial distribution, with expected value $np$ and variance $np(1-p)$. Its exact probability mass function and its normalized count areHere $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.

At $n=100$, $K_n$ has expected value $20$ and standard deviation $4$. The exact binomial distribution gives $\mathbb{P}(16\le K_n\le 24)=0.740141\dots$. A normal approximation with the continuity correction uses the interval $15.5$ to $24.5$:The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.

The strong law of large numbers and weak law of large numbers locate the average: ${\overline{X}}_n\to \mu$, respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is $N(\mu ,{\sigma }^2/n)$, with standard error $\sigma /\sqrt{n}$. This explains why four times as many independent observations halve the standard error. When $\sigma$ is known, ${\overline{X}}_n\pm 1.96\sigma /\sqrt{n}$ has approximately $95\%$ coverage for $\mu$ at large $n$, giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.

A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when $0<p<1$.^[ref]

There are three useful limits to this picture:

A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.^[ref]

A brief proof outline uses the characteristic function of $Y=(X_1-\mu )/\sigma$. Its expected value is zero and its variance is one, so the finite second moment givesThe Taylor expansion of the complex exponential, with its remainder controlled by $Y^2$, and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of $Z_n$:The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.

Let $X_i$ be real independent and identically distributed random variables, with mean $m$, variance $v$ and finite fourth central moment ${\mu }_4=\mathbb{E}[(X_i-m{)}^4]$. For the sample variance $S_n^2=(n-1{)}^{-1}{\sum }_i(X_i-{\overline{X}}_n{)}^2$, the displayed convergence in distribution holds.

Indeed $n^{-1}{\sum }_i(X_i-{\overline{X}}_n{)}^2=n^{-1}{\sum }_i(X_i-m{)}^2-({\overline{X}}_n-m{)}^2$. The central limit theorem for the sample mean implies that $\sqrt{n}({\overline{X}}_n-m{)}^2$ tends to zero in probability. Changing the divisor from $n$ to $n-1$ also has negligible effect at this scale, by the law of large numbers. Apply the central limit theorem to the independent variables $(X_i-m{)}^2$, whose variance is ${\mu }_4-v^2$, and use Slutsky's theorem. Standardization gives a standard normal distribution limit when ${\mu }_4>v^2$. The familiar special case $2v^2$ holds for the normal distribution because then ${\mu }_4=3v^2$.

Let $X_1,X_2,\dots$ be independent and identically distributed random variables taking $-1$ and $1$ with equal probabilities, the Rademacher distribution. The two opposite complex exponentials give ${\phi }_X(t)=(e^{it}+e^{-it})/2=\cos t$. Their expected value is zero and their variance is one. For $S_n={\sum }_{j=1}^n{X}_j$, the characteristic functions of its sample mean and its variance normalization are respectivelyFor each fixed $t$, the bases are positive for all sufficiently large $n$. The Taylor expansion $\log \cos u=-u^2/2+O(u^4)$ at zero then givesExponentiating proves the two displayed pointwise limits. The first limit is the characteristic function of the constant zero; the second is that of the standard normal distribution. The Lévy continuity theorem gives the corresponding convergence in distribution statements. Averaging has variance $1/n$, whereas dividing the sum by $\sqrt{n}$ preserves variance one. This sign calculation illustrates why the law of large numbers and the central limit theorem use different scales; it is not a proof for arbitrary probability distributions.

Suppose Lévy processes $X^n$ converge to $X$ in probability at every fixed time, andThen $X$ starts at zero, has independent increments and stationary increments, and has stochastic continuity. Finite increment vectors inherit their independent laws by convergence in distribution; the near-zero estimate supplies continuity at zero. Hence $X$ has a càdlàg modification that is a Lévy process. The hypotheses alone cannot assert that the original version has càdlàg paths.

A confidence band provides simultaneous bounds on an unknown function over its entire index set. Its coverage event is the intersection over all indices, unlike separate pointwise confidence intervals. For a continuous cumulative distribution function, the Kolmogorov-Smirnov theorem gives an asymptotic band $F_n\pm c_{\alpha }/\sqrt{n}$, clipped to $[0,1]$, where $c_{\alpha }$ is a $(1-\alpha )$-quantile of the absolute Brownian bridge maximum. The simultaneous event equals $\sqrt{n}\Vert F_n-F{\Vert }_{\infty }\le c_{\alpha }$, so convergence in distribution at this continuity point proves coverage tending to $1-\alpha$.

Suppose $\theta >0$ and a positive estimator $T_n$ satisfies $\sqrt{n}(T_n-\theta )/\theta \stackrel{d}{\to }N(0,1)$. Choose the standard normal quantile $c={\Phi }^{-1}(1-\alpha /2)$. For $n>c^2$, an asymptotic confidence interval of level $1-\alpha$ isProof. Convergence in distribution gives $\mathbb{P}(-c<\sqrt{n}(T_n-\theta )/\theta <c)\to 1-\alpha$. Multiplication by the positive $\theta$ and rearrangement makes this event equivalent to $\theta (1-c/\sqrt{n})<T_n<\theta (1+c/\sqrt{n})$. Both factors are positive when $n>c^2$, so division yields exactly the displayed interval. Thus its coverage probability tends to $1-\alpha$. For $95\%$ coverage, $c\approx 1.96$. This inversion retains the unknown scale $\theta$ in the normalized error, rather than replacing it by the observed estimator before inversion.

Every characteristic function is uniformly continuous and satisfies $\phi (0)=1$ and $|\phi (t)|\le 1$. Continuity follows from the dominated convergence theorem applied to $e^{itX}$.

More explicitly, if $t_n\to t$, then $e^{i{t}_{n}X}\to e^{itX}$ pointwise and $|e^{i{t}_{n}X}|=1$. Apply the dominated convergence theorem to the real part and imaginary part separately. There is no finite moment hypothesis. For uniform continuity, the stronger estimatefollows by factoring out $e^{itX}$, whose absolute value is one. The integrand tends pointwise to zero and is bounded by two, so the dominated convergence theorem applies again. The bound is independent of $t$, which supplies uniform continuity on the entire real line.^[ref]

If the first absolute moment is finite, $|e^{iu}-1|\le |u|$ additionally gives $|{\phi }_X(t+h)-{\phi }_X(t)|\le |h|\mathbb{E}[|X|]$, a Lipschitz continuity estimate. The earlier argument matters because continuity survives even when this expected value is infinite. For convergence in distribution, the separate Lévy continuity theorem concerns limits of whole sequences of characteristic functions; the argument here establishes continuity of one characteristic function.

If $X_n\stackrel{d}{\to }c$ for fixed real $c$, convergence of the distribution functions at $c-\epsilon$ and $c+\epsilon$ shows $\mathbb{P}(|X_n-c|>\epsilon )\to 0$. Thus convergence in distribution to a deterministic limit implies convergence in probability to that limit, even though the implication fails for nonconstant limits on a specified probability space.

For real random variables on one probability space, fix a continuity point $x$ of the cumulative distribution function $F_X$. For every $\epsilon >0$, inclusions of events giveTake the lower and upper limits as $n\to \infty$, using convergence in probability, and then let $\epsilon \downarrow 0$. Continuity at $x$ makes both bounds $F_X(x)$, proving convergence in distribution. The two modes of convergence have different information: convergence in probability concerns the coupling on one probability space, whereas convergence in distribution concerns only the marginal probability distributions.

The main modes include almost sure convergence, convergence in probability, convergence in Lp, and convergence in distribution. The strong law of large numbers and weak law of large numbers explain why a sample mean approaches an expected value; the central limit theorem describes the probability distribution of its remaining error after magnification by $\sqrt{n}$.

Every non-degenerate limit in convergence in distribution of positively scaled and centered sample maxima is a max-stable distribution. Up to positive affine transformations its law is the Gumbel distribution, a Fréchet distribution with positive shape parameter, or a negative Weibull distribution with positive shape parameter. This classifies possible limits, without asserting that every distribution function admits a limiting normalization.

Extreme value theory studies limiting probability distributions of normalized sample maxima and related extremes. The standard limits for maxima include the Fréchet distribution, negative Weibull distribution, and Gumbel distribution.

The question is often practical: if a batch finishes only when its slowest task finishes, its completion time is a sample maximum. A typical individual delay or a sample mean does not answer that question. The central limit theorem describes suitably normalized sums and sample means under its assumptions; it does not give a limit law for sample maxima. Extreme value theory instead describes how the largest observation changes as the number of opportunities for a large observation grows.

For independent and identically distributed random variables $X_1,X_2,\dots$ with distribution function $F$, define the sample maximumThe event $M_n\le x$ means that every observation is at most $x$, so independence givesExtreme value theory asks whether there are constants $a_n>0$ and $c_n\in \mathbb{R}$ for which $(M_n-c_n)/a_n$ has convergence in distribution to a non-degenerate probability distribution. Equivalently, $F(c_n+a_{n}z{)}^n$ must tend to a limiting distribution function at its continuity points. The extremal types theorem says that any such limit belongs, up to a change of location and positive scale, to the Gumbel distribution, Fréchet distribution, or negative Weibull distribution. It does not guarantee that a limit exists for every $F$. ^[ref].

The maximum of independent exponential random variables gives a particularly transparent worked example: its center grows as $(\log n)/\lambda$, and its fluctuations have a Gumbel distribution limit on the scale $1/\lambda$. Here and below $\log$ denotes the natural logarithm. This normalization depends on the exponential distribution. Other survival functions can lead to the Fréchet distribution or negative Weibull distribution, with different constants $a_n,c_n$; the Gumbel distribution and the exponential example's normalization are not universal.

Independence is essential to the product calculation. If instead $X_1=X_2=\cdots =X$ for one rate-$\lambda$ exponential random variable, then $M_n=X$ for every $n$. Its distribution function stays $1-e^{-\lambda x}$ for $x\ge 0$ and zero below zero, and its median stays $(\log 2)/\lambda$. Perfect dependence creates no logarithmic growth: repeating the same random variable supplies no new chance of a larger value.

A historical starting point is R. A. Fisher and L. H. C. Tippett's 1928 paper on limiting laws of the largest or smallest sample member, published in volume 24, issue 2, pages 180–190 of the Proceedings of the Cambridge Philosophical Society. Its abstract studies the possible limiting forms and emphasizes that approach to a limit can be slow. ^[ref].

If nonnegative integer-valued random variables $X_n$ have all fixed factorial moments converging to ${\lambda }^k$, then their convergence in distribution is to the Poisson distribution with mean $\lambda$. The first-moment bound gives uniform tightness, and bounds on higher moments give uniform integrability of each fixed lower moment. Every subsequential limit therefore has the indicated factorial moments. For such a limit $X$ and $t\ge 0$, the nonnegative binomial expansion and the monotone convergence theorem give $\mathbb{E}(1+t{)}^X={\sum }_{k\ge 0}{t}^k{\lambda }^k/k!=e^{\lambda t}$. This identifies its moment-generating function as that of the Poisson distribution, uniquely determining the limit.

The standard Gumbel distribution for maxima has distribution function $\Lambda (z)=e^{-e^{-z}}$ for every real $z$. Its probability density function is the derivative of its distribution function:This article uses the maxima convention throughout. Reflecting a standard Gumbel distribution random variable $Z$ to $-Z$ gives the minima convention, with distribution function $1-\exp (-e^z)$. The two conventions have opposite directions of asymmetry, so their formulas must not be interchanged. ^[ref].

For a location $\mu \in \mathbb{R}$ and positive scale $b>0$, let $Y=\mu +bZ$. This location-scale family has cumulative distribution function and probability density functionSolving $\Lambda (z)=p$ gives the standard quantile function $q(p)=-\log (-\log p)$, $0<p<1$, and hence the standard medianThe median of $Y$ is therefore $\mu -b\log (\log 2)$. Location $\mu$ is not the median: at $x=\mu$ the cumulative distribution function is $e^{-1}\approx 0.367879$.

To derive the Gumbel distribution as an extreme-value limit, take the maximum of independent exponential random variables with rate $\lambda >0$ and setThe exact distribution function isThe finite-$n$ lower boundary is required because $M_n\ge 0$. For each fixed real $z$, eventually $z>-\log n$, and the natural logarithm of the first branch satisfiesExponentiating proves convergence in distribution to the standard Gumbel distribution. It is the familiar limit $(1-u/n{)}^n\to e^{-u}$ with $u=e^{-z}$: a simple calculation reveals a whole limiting shape. The name honors Emil Gumbel; Siegrist gives this exponential-maxima derivation and the standard Gumbel distribution formulas. ^[ref].

For the toy delay model $\lambda =1{\mathrm{s}}^{-1}$, $n=100$, the limiting Gumbel distribution approximation to $M_{100}$ has $\mu =(\log 100)\mathrm{s}$ and $b=1\mathrm{s}$. Its predicted median iscompared with the exact median $-\log (1-2^{-1/100})\mathrm{s}\approx 4.975147\mathrm{s}$. The approximation is about $0.003464$ seconds lower in this example. It is not an exact finite-sample probability distribution: the approximating Gumbel distribution assigns positive probability to negative delays, while the exact sample maximum cannot be negative.

The maximum domain of attraction of the Gumbel distribution is characterized by an exponential tail ratio on the scale of a Gumbel auxiliary function. This allows many original probability distributions to share the limiting shape, but it does not make the centering $\log n$ appropriate to all of them.

For the Gamma distribution with fixed positive integer shape $m$ and rate one, the survival function is $e^{-t}{\sum }_{j=0}^{m-1}{t}^j/j!$. Its sample maximum centered by $\log n+(m-1)\log \log n-\log ((m-1)!)$ has convergence in distribution to the standard Gumbel distribution, without an additional scale factor.

If the characteristic functions ${\phi }_n$ of probability distributions ${\mu }_n$ converge pointwise to a function $\phi$ that is continuous at zero, then $\phi$ is a characteristic function and ${\mu }_n$ converges in distribution to its distribution. Conversely, convergence in distribution implies pointwise convergence of the characteristic functions.

The probability distribution approached in convergence in distribution. It can depend on the initial state of a Markov chain when several closed communicating classes are accessible.

A distribution function $F$ lies in the maximum domain of attraction of $G$ if $F(a_{n}x+b_n{)}^n\to G(x)$ at every continuity point of $G$ for some $a_n>0$ and real $b_n$. This expresses convergence in distribution of normalized sample maxima.

Let $n\ge 1$ be an integer and let $X_1,\dots ,X_n$ be independent and identically distributed random variables with an exponential distribution of rate $\lambda >0$. Their sample maximum $M_n$ is nonnegative. Substituting the exponential distribution's cumulative distribution function into the distribution functions of independent minima and maxima gives the exact lawFor example, at $x=(\log 2)/\lambda$ each observation is below the threshold with probability $1/2$, but all $n$ observations are below it with probability $2^{-n}$. Thus an individual observation's median becomes an increasingly unlikely upper bound for the whole batch. The general order statistic density in Siegrist's treatment, with $k=n$, also gives this law on integration. ^[ref].

Solving $(1-e^{-\lambda x}{)}^n=p$ gives the exact quantile function, for $0<p<1$,In a toy model with $\lambda =1{\mathrm{s}}^{-1}$, the median maximum is $0.693147$ seconds for $n=1$, $2.703555$ seconds for $n=10$, and $4.975147$ seconds for $n=100$. All three use the exact distribution function; the increase comes from taking a maximum over more observations. The $n=2$ case connects to the existing maximum of two exponentials as a scaled sum.

The Gumbel distribution explains the logarithmic shift visible here: after subtracting $(\log n)/\lambda$ and measuring deviations in units of $1/\lambda$, the distribution functions approach one fixed shape. The next derivation makes that convergence in distribution explicit.