Central limit theorem ($\frac{S_n-n\mu }{\sigma \sqrt{n}}\stackrel{d}{\to }N(0,1)$)

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with expected value $\mu =\mathbb{E}[X_1]$ and finite, strictly positive variance ${\sigma }^2=\mathbb{E}[(X_1-\mu {)}^2]$. Write $S_n={\sum }_{j=1}^n{X}_j$ and ${\overline{X}}_n=S_n/n$ for their sum and sample mean. The central limit theorem statesPrecisely, for every real $z$,where $\Phi$ is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals $\mu$ almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.

The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: $\mathbb{E}[S_n]=n\mu$. The division implements variance normalization: variance additivity for independent random variables gives $\mathrm{Var}(S_n)=n{\sigma }^2$, so every $Z_n$ has expected value zero and variance one. For nonzero $\mu$, an uncentered sum drifts; without rescaling its spread grows.

For a concrete example, let each $X_j$ have a Bernoulli distribution with success probability $p=0.2$. Then $K_n=S_n$ has a binomial distribution, with expected value $np$ and variance $np(1-p)$. Its exact probability mass function and its normalized count areHere $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.

At $n=100$, $K_n$ has expected value $20$ and standard deviation $4$. The exact binomial distribution gives $\mathbb{P}(16\le K_n\le 24)=0.740141\dots$. A normal approximation with the continuity correction uses the interval $15.5$ to $24.5$:The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.

The strong law of large numbers and weak law of large numbers locate the average: ${\overline{X}}_n\to \mu$, respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is $N(\mu ,{\sigma }^2/n)$, with standard error $\sigma /\sqrt{n}$. This explains why four times as many independent observations halve the standard error. When $\sigma$ is known, ${\overline{X}}_n\pm 1.96\sigma /\sqrt{n}$ has approximately $95\%$ coverage for $\mu$ at large $n$, giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.

A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when $0<p<1$.^[ref]

There are three useful limits to this picture:

A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.^[ref]

A brief proof outline uses the characteristic function of $Y=(X_1-\mu )/\sigma$. Its expected value is zero and its variance is one, so the finite second moment givesThe Taylor expansion of the complex exponential, with its remainder controlled by $Y^2$, and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of $Z_n$:The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.

Let $X_k$ be independent and identically distributed random variables with expected value zero, variance one, and a moment-generating function finite in a neighborhood of zero. Each row of deterministic real weights $a_{n,k}$ is finite. If ${\max }_k|a_{n,k}|\to 0$ and ${\sum }_k{a}_{n,k}^2\to v<\infty$, then the weighted sums converge in distribution to a normal distribution with mean zero and variance $v$; $v=0$ denotes the constant zero.

By the quadratic expansion of a cumulant-generating function, write $K(u)=u^2/2+r(u)$ with $r(u)=o(u^2)$. For a fixed real $t$, put ${\eta }_n={\sup }_{0<|u|\le |t|{\max }_k|a_{n,k}|}|r(u)|/u^2$, with the supremum taken as zero if its range is empty. Then ${\eta }_n\to 0$ andThus the log moment-generating function of the sum tends to $v{t}^2/2$. The convergence theorem for moment-generating functions proves the conclusion. The supremum is the step that makes the remainder estimate uniform across the row; a separate little-o assertion for each term would not justify summing an increasing number of terms.

For each $n$, let $Z_{n,i}$ be independent centered random variables. If the sum of their variances tends to ${\sigma }^2>0$ and ${\sum }_i\mathbb{E}[Z_{n,i}^2{1}_{\{|Z_{n,i}|>\epsilon \}}]\to 0$ for every $\epsilon >0$, their sum converges in distribution to the centered normal distribution with variance ${\sigma }^2$. The second hypothesis is the Lindeberg condition. If the total variance tends to zero, the corresponding degenerate limit follows directly from the Chebyshev inequality.

For independent centered random variables with total variance $s_n^2$, the Lyapunov condition requires the displayed expression to tend to zero for some $\delta >0$. The inequality $z^2{1}_{\{|z|>\epsilon s_n\}}\le |z{|}^{2+\delta }/(\epsilon s_n{)}^{\delta }$ implies the Lindeberg condition after normalization. The Lindeberg-Feller central limit theorem then makes the normalized sum converge to the standard normal distribution.

For independent identically distributed mean-zero variables with finite positive variance, the central limit theorem gives a positive eventual lower bound for $\mathbb{P}(S_n/\sqrt{n}\ge K)$ for every finite $K$. The decreasing events ${\bigcup }_{n\ge m}\{S_n/\sqrt{n}\ge K\}$ have probabilities bounded below, so continuity from above of a measure gives positive probability that the limsup is at least $K$. The limsup itself is unchanged by removing any finite initial sum, hence is tail measurable. Kolmogorov zero-one law makes each event “limsup at least $K$” certain; intersect over integer $K$. The result is false for zero variance. One should apply the zero-one law to the limsup, rather than assume the exact threshold infinitely-often event is unaffected by a vanishing change.

A normal approximation replaces a sampling distribution by a normal distribution, often through a central limit theorem. In a Wald test, approximate statistical power is determined by the standardized difference under the alternative. This yields Neyman allocation for fixed treatment-specific variances.

A continuity correction shifts a discrete probability boundary by half a lattice spacing before using a continuous normal approximation. For an upper-tail event $T\ge t$ on a unit lattice, the approximation is $1-\Phi ((t-\frac{1}{2}-E(T))/\sqrt{\mathrm{Var}(T)})$. Lower tails use the boundary $t+\frac{1}{2}$. The correction is used in approximate Wilcoxon signed-rank tests and Wilcoxon rank sum tests; tied ranks also require the appropriate variance adjustment.

The Lindeberg condition requires the contribution of unusually large summands in a triangular array to become negligible compared with the array's total variance. Together with convergence of that variance, it gives the Lindeberg central limit theorem.

For independent identically distributed centered random variables in a separable Hilbert space with finite second moment, $n^{-1/2}{\sum }_{i=1}^n{X}_i$ converges in distribution to a centered Gaussian random element with the same covariance operator.

For independent identically distributed $Z_i$ with mean zero and variance one,The numerator obeys the central limit theorem, the denominator converges almost surely to one by the strong law, and Slutsky's theorem combines them.

For independent identically distributed increments of mean zero and variance one, the diffusively rescaled, linearly interpolated random walk converges weakly in $C[0,1]$ to Brownian motion.

For centered unit-variance independent and identically distributed increments with $m_4=\mathbb{E}{\xi }^4<\infty$, an interpolated increment is $N^{-1/2}{\sum }_j{c}_j{\xi }_j$. Here $c_j$ is the length of the overlap of $(Ns,Nt)$ with the $j$th unit cell, so $0\le c_j\le 1$ and $\sum c_j=N(t-s)$. Independence gives its fourth moment as $N^{-2}[3(\sum c_j^2{)}^2+(m_4-3)\sum c_j^4]$. This is at most $\max (3,m_4)N^{-2}(\sum c_j^2{)}^2$, and $\sum c_j^2\le N(t-s)$ gives the bound, including intervals crossing a mesh point. A uniform fourth-moment continuity criterion yields tightness of probability measures on $C_0[0,1]$.

For a random walk with centered independent identically distributed increments of variance one, the maximum of its polygonal interpolation is the maximum at its grid vertices. The maximum functional is Lipschitz for the supremum norm, so the continuous mapping theorem applied to the Donsker invariance principle proves the displayed convergence in distribution. The Brownian reflection principle then gives upper-tail limits $2(1-\Phi (a))$ at all $a>0$, since the limiting maximum has no atom there.

For a random walk with independent identically distributed steps of mean zero and variance one, the Donsker invariance principle and the continuous mapping theorem for integration give this limit. The integral of the polygonal interpolation differs from the right-endpoint sum by $S_n/(2n^{3/2})$, whose squared expectation is $1/(4n^2)$. The Slutsky theorem removes that error. The limiting variable is the time-one value of integrated Brownian motion, with law $N(0,1/3)$.

For independent identically distributed random vectors $Z_i$ with mean $\mu$ and finite covariance matrix $V$,