= Central limit theorem
{title2=$\frac{S_n-n\mu}{\sigma\sqrt n}\xrightarrow{d}N(0,1)$}
{wiki}
= CLT
{c}
{synonym}
Let $X_1,X_2,\ldots$ be real <independent and identically distributed random variables> with <expected value> $\mu=\mathbb E[X_1]$ and finite, strictly positive <variance> $\sigma^2=\mathbb E[(X_1-\mu)^2]$. Write $S_n=\sum_{j=1}^nX_j$ and $\overline X_n=S_n/n$ for their sum and <sample mean>. The <central limit theorem> states
$$
Z_n=\frac{S_n-n\mu}{\sigma\sqrt n}
=\frac{\sqrt n(\overline X_n-\mu)}{\sigma}
\xrightarrow{d}N(0,1).
$$
Precisely, for every real $z$,
$$
\mathbb P(Z_n\leq z)\longrightarrow\Phi(z),
\qquad
\Phi(z)=\frac1{\sqrt{2\pi}}\int_{-\infty}^z e^{-u^2/2}\,du,
$$
where $\Phi$ is the <standard normal distribution function>. The conclusion is <convergence in distribution> to the <standard normal distribution>. If the <variance> is zero, every observation equals $\mu$ <almost surely>; the displayed normalization is then undefined and there is no fluctuating error to describe.
The striking feature is that the observations need not have a <normal distribution>. They may be binary, bounded, or strongly skewed. Their detailed <probability distribution> disappears from this limit except through the <expected value> and <variance>. The subtraction removes predictable growth: $\mathbb E[S_n]=n\mu$. The division implements <variance normalization>: <variance additivity for independent random variables> gives $\operatorname{Var}(S_n)=n\sigma^2$, so every $Z_n$ has <expected value> zero and <variance> one. For nonzero $\mu$, an uncentered sum drifts; without rescaling its spread grows.
For a concrete example, let each $X_j$ have a <Bernoulli distribution> with success <probability> $p=0.2$. Then $K_n=S_n$ has a <binomial distribution>, with <expected value> $np$ and <variance> $np(1-p)$. Its exact <probability mass function> and its normalized count are
$$
\mathbb P(K_n=k)=\binom nk p^k(1-p)^{n-k},\quad k=0,\ldots,n,
\qquad
Z_n=\frac{K_n-np}{\sqrt{np(1-p)}}.
$$
Here $\binom nk$ is a <binomial coefficient>. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same <standard normal distribution> as sums of fair trials.
\Image[/central-limit-distributions.png]
{title=Exact standardized binomial distributions approach the standard normal curve}
{description=For <independent> <Bernoulli trials> with $p=0.2$, each blue bar has width equal to the normalized lattice spacing and area equal to the exact <binomial distribution> mass at its center. Heights are mass divided by spacing, so they can be compared with the orange <probability density function> of the <standard normal distribution>. All four normalized counts have <expected value> zero and <variance> one. The rectangles visualize discrete masses; they do not make the discrete <probability distributions> continuous.}
At $n=100$, $K_n$ has <expected value> $20$ and <standard deviation> $4$. The exact <binomial distribution> gives $\mathbb P(16\leq K_n\leq24)=0.740141\ldots$. A <normal approximation> with the <continuity correction> uses the interval $15.5$ to $24.5$:
$$
\mathbb P(16\leq K_n\leq24)
\approx\Phi\left(\frac{24.5-20}{4}\right)-\Phi\left(\frac{15.5-20}{4}\right)
=0.739411\ldots.
$$
The small discrepancy is an approximation error, not an equality supplied by the <central limit theorem>.
The <strong law of large numbers> and <weak law of large numbers> locate the average: $\overline X_n\to\mu$, respectively by <almost sure convergence> and <convergence in probability>. The <central limit theorem> magnifies that shrinking error and describes its shape. In terms of the <sample mean>, the resulting <normal approximation> is $N(\mu,\sigma^2/n)$, with <standard error> $\sigma/\sqrt n$. This explains why four times as many <independent> observations halve the <standard error>. When $\sigma$ is known, $\overline X_n\pm1.96\sigma/\sqrt n$ has approximately $95\%$ coverage for $\mu$ at large $n$, giving a <confidence interval>. This coverage refers to repeated samples, not a guarantee for an individual interval.
A useful application is the <Monte Carlo method>: estimate a success <probability> by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The <strong law of large numbers> justifies convergence of such proportions for <independent> repetitions; the <central limit theorem> supplies their large-sample error scale when $0<p<1$.https://hedibert.org/wp-content/uploads/2013/12/1949MetropolisUlam.pdf[Metropolis and Ulam, The Monte Carlo Method, JASA 44 (1949), 335–341, especially pp. 336–337]{ref}
There are three useful limits to this picture:
* A <heavy-tailed distribution> can have infinite <variance>, so the displayed hypothesis must be checked. A <Pareto distribution> with minimum one and shape $3/2$ has finite <expected value> three but infinite <variance>. The <strong law of large numbers> still applies, but the stated <central limit theorem> does not. More dramatically, the <Cauchy sample mean has an unchanged distribution>: even averaging need not concentrate when the <expected value> is undefined. A <heavy-tailed distribution> with finite <variance> is allowed by the stated <central limit theorem>.
* <Independence> matters. If all $X_j$ equal the same nonconstant <Bernoulli random variable>, then $S_n=nX_1$ and $\overline X_n=X_1$. There is no accumulation of new information, and the <variance> of the sum grows like $n^2$, not $n$. Other versions of the <central limit theorem> can allow dependence, but require hypotheses controlling it.
* <Convergence in distribution> is a limiting assertion, with no universal sample size or error rate under finite <variance> alone. It does not automatically give accurate relative estimates of very small tail <probabilities>. Nor does it mean that discrete point masses converge as <probability density functions>: a <binomial distribution> remains discrete for every finite $n$. If the success <probability> changes with $n$ so that $np_n\to\lambda$ and $p_n\to0$, the relevant result is the <Poisson limit theorem>, rather than the fixed-$p$ argument above.
A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The <Lindeberg condition> and <Lindeberg-Feller central limit theorem> below describe a version for sums whose summands need not share a <probability distribution>.https://doi.org/10.1007/BF01494395[Lindeberg, Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung (1922)]{ref}
A brief proof outline uses the <characteristic function> of $Y=(X_1-\mu)/\sigma$. Its <expected value> is zero and its <variance> is one, so the finite <second moment> gives
$$
\varphi_Y(u)=1-\frac{u^2}{2}+o(u^2)\quad(u\to0).
$$
The <Taylor expansion> of the complex exponential, with its remainder controlled by $Y^2$, and the <dominated convergence theorem> justify this step without a third-moment assumption. <Independence> then gives the <characteristic function> of $Z_n$:
$$
\varphi_{Z_n}(t)=\left(\varphi_Y(t/\sqrt n)\right)^n
=\left(1-\frac{t^2}{2n}+o(n^{-1})\right)^n\longrightarrow e^{-t^2/2}.
$$
The limit is the <characteristic function> of the <standard normal distribution>; the <Lévy continuity theorem> turns this into <convergence in distribution>.
Back to article page
