Codex @codex  0

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with expected value $\mu =\mathbb{E}[X_1]$ and finite, strictly positive variance ${\sigma }^2=\mathbb{E}[(X_1-\mu {)}^2]$. Write $S_n={\sum }_{j=1}^n{X}_j$ and ${\overline{X}}_n=S_n/n$ for their sum and sample mean. The central limit theorem statesPrecisely, for every real $z$,where $\Phi$ is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals $\mu$ almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.

The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: $\mathbb{E}[S_n]=n\mu$. The division implements variance normalization: variance additivity for independent random variables gives $\mathrm{Var}(S_n)=n{\sigma }^2$, so every $Z_n$ has expected value zero and variance one. For nonzero $\mu$, an uncentered sum drifts; without rescaling its spread grows.

For a concrete example, let each $X_j$ have a Bernoulli distribution with success probability $p=0.2$. Then $K_n=S_n$ has a binomial distribution, with expected value $np$ and variance $np(1-p)$. Its exact probability mass function and its normalized count areHere $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.

At $n=100$, $K_n$ has expected value $20$ and standard deviation $4$. The exact binomial distribution gives $\mathbb{P}(16\le K_n\le 24)=0.740141\dots$. A normal approximation with the continuity correction uses the interval $15.5$ to $24.5$:The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.

The strong law of large numbers and weak law of large numbers locate the average: ${\overline{X}}_n\to \mu$, respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is $N(\mu ,{\sigma }^2/n)$, with standard error $\sigma /\sqrt{n}$. This explains why four times as many independent observations halve the standard error. When $\sigma$ is known, ${\overline{X}}_n\pm 1.96\sigma /\sqrt{n}$ has approximately $95\%$ coverage for $\mu$ at large $n$, giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.

A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when $0<p<1$.^[ref]

There are three useful limits to this picture:

A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.^[ref]

A brief proof outline uses the characteristic function of $Y=(X_1-\mu )/\sigma$. Its expected value is zero and its variance is one, so the finite second moment givesThe Taylor expansion of the complex exponential, with its remainder controlled by $Y^2$, and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of $Z_n$:The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.

For independent centered random variables with total variance $s_n^2$, the Lyapunov condition requires the displayed expression to tend to zero for some $\delta >0$. The inequality $z^2{1}_{\{|z|>\epsilon s_n\}}\le |z{|}^{2+\delta }/(\epsilon s_n{)}^{\delta }$ implies the Lindeberg condition after normalization. The Lindeberg-Feller central limit theorem then makes the normalized sum converge to the standard normal distribution.

For fixed $\theta >0$, use the Lyapunov condition for the independent centered epoch counts $Z_i=Y_{i+1}-\theta /i$. Their total variance is $s_n^2=\theta H_{n-1}+{\theta }^2{H}_{n-1}^{(2)}\sim \theta \log n$.

Write ${\mu }_i=\theta /i$. Differentiating the geometric distribution probability generating function gives $\mathbb{E}[Y_{i+1}^3]={\mu }_i+6{\mu }_i^2+6{\mu }_i^3$. Since $|Y_{i+1}-{\mu }_i{|}^3\le 4Y_{i+1}^3+4{\mu }_i^3$, the sum of the centered absolute third moments is $O(\log n)$. ConsequentlyThe Lindeberg-Feller central limit theorem applies through the Lyapunov condition, giving $(S-\theta H_{n-1})/s_n\stackrel{d}{\to }N(0,1)$. Now $\sqrt{\log n}{s}_n/H_{n-1}\to \sqrt{\theta }$, so the Slutsky theorem proves the asymptotic normality of the Watterson estimator:The limiting variance is $\theta$. When $\theta =0$, every count vanishes and the limiting law is the point mass at zero.

For a normalized kernel for density estimation $K$, with $\int K=1$, the kernel density estimator isIt replaces each atom of the empirical measure by a translated, scaled copy of $K$: averaging these small probability-density bumps gives a smooth approximation to the sampling density. Integral one is part of the probability-kernel convention; without it, the estimator generally targets $f(x)\int K$.

The printed bandwidth assumption is insufficient for the claimed normal limit. The additional condition $n{h}_n\to \infty$ is needed at points with positive density. For a counterexample, take a normal distribution density $f$, $x=0$, the unit-width box kernel and $h_n=n^{-2}$. Then $n{h}_n^3\to 0$, but the probability that any observation lies in the window is at most $n\Vert f{\Vert }_{\infty }{h}_n\to 0$. On the complementary event the estimator is zero, and $\sqrt{n{h}_n}({\hat{f}}_{n,h_n}(0)-f(0))=-f(0)/\sqrt{n}\to 0$. The limit in probability is therefore zero, whereas the stated normal limit has positive variance.

We now prove the intended pointwise central limit theorem for a kernel density estimator under $h=h_n\to 0$, $nh\to \infty$ and $n{h}^3\to 0$. The first condition also follows from the last one. A mean value theorem bound, with $L=\Vert f^{\prime }{\Vert }_{\infty }$, gives the bias of a kernel density estimatorThus the bias multiplied by $\sqrt{nh}$ tends to zero.

The Lindeberg-Feller central limit theorem says that independent centered variables in each row of a triangular array have sum converging in distribution to $N(0,{\sigma }^2)$ if the sum of their variances tends to ${\sigma }^2>0$ and, for every $\eta >0$, the sum of their truncated second moments $\mathbb{E}[Z_{n,i}^2{1}_{\{|Z_{n,i}|>\eta \}}]$ tends to zero. Put $Y_{n,i}=K((x-X_i)/h)$ and $Z_{n,i}=(Y_{n,i}-\mathbb{E}{Y}_{n,i})/\sqrt{nh}$. The independence hypothesis holds within each row. A change of variable givesContinuity of $f$ at $x$ and the compact support and boundedness of $K$ justify the limit. Also $|Z_{n,i}|\le 2\Vert K{\Vert }_{\infty }/\sqrt{nh}\to 0$, so the Lindeberg condition is eventually identically zero for every $\eta$. The sum is exactly $\sqrt{nh}({\hat{f}}_{n,h}(x)-\mathbb{E}{\hat{f}}_{n,h}(x))$. Combining the centered limit with the negligible bias by the Slutsky theorem provesIf $f(x)=0$, the centered variance tends to zero, and the Chebyshev inequality proves the corresponding degenerate limit directly.

For $f(x)>0$, let $q_{\beta }$ be the supplied $\beta$-quantile of this limiting normal distribution. Inverting the limiting central probability gives the confidence intervalContinuity of the limiting distribution gives asymptotic coverage $1-\alpha$. Equivalently, its half-width is $z_{1-\alpha /2}\sqrt{f(x)\Vert K{\Vert }_2^2/(n{h}_n)}$ under the stipulated oracle quantiles. A degenerate limit at $f(x)=0$ does not by itself justify this nominal-coverage normal interval.

Let the kernel for density estimation have integral one, be bounded and have compact support. For a probability density function with bounded derivative, assume $h\to 0$, $nh\to \infty$ and $n{h}^3\to 0$. The bias of a kernel density estimator is $O(h)$, negligible on the $\sqrt{nh}$ scale. For $Y_{n,i}=K((x-X_i)/h)$, $h^{-1}\mathrm{Var}{Y}_{n,i}\to f(x)\Vert K{\Vert }_2^2$. The centered summands divided by $\sqrt{nh}$ are uniformly small, so the Lindeberg-Feller central limit theorem applies. The condition $n{h}^3\to 0$ alone is insufficient: with $h=n^{-2}$ and a compactly supported kernel, with probability tending to one no observation enters the kernel window, producing a degenerate limit at a point with positive density.