Central limit theorem Created 2026-09-24 Updated 2026-10-14
Let be real independent and identically distributed random variables with expected value and finite, strictly positive variance . Write and for their sum and sample mean. The central limit theorem states
Precisely, for every real ,
where is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.
The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: . The division implements variance normalization: variance additivity for independent random variables gives , so every has expected value zero and variance one. For nonzero , an uncentered sum drifts; without rescaling its spread grows.
For a concrete example, let each have a Bernoulli distribution with success probability . Then has a binomial distribution, with expected value and variance . Its exact probability mass function and its normalized count are
Here is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.
Figure 1.
Exact standardized binomial distributions approach the standard normal curve
. For independent Bernoulli trials with , each blue bar has width equal to the normalized lattice spacing and area equal to the exact binomial distribution mass at its center. Heights are mass divided by spacing, so they can be compared with the orange probability density function of the standard normal distribution. All four normalized counts have expected value zero and variance one. The rectangles visualize discrete masses; they do not make the discrete probability distributions continuous. Details.
At , has expected value and standard deviation . The exact binomial distribution gives . A normal approximation with the continuity correction uses the interval to :
The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.
The strong law of large numbers and weak law of large numbers locate the average: , respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is , with standard error . This explains why four times as many independent observations halve the standard error. When is known, has approximately coverage for at large , giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.
A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when .[ref]
There are three useful limits to this picture:
A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.[ref]
A brief proof outline uses the characteristic function of . Its expected value is zero and its variance is one, so the finite second moment gives
The Taylor expansion of the complex exponential, with its remainder controlled by , and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of :
The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.
A nondegenerate distribution function is a max-stable distribution if, for every integer , there are and such that
Equivalently, the suitably centered and positively scaled sample maximum of independent and identically distributed random variables with law again has law . The extremal types theorem states that every nondegenerate limiting law of normalized sample maxima is a max-stable distribution. Thus this property identifies all possible limiting types, although a particular starting distribution function need not admit any such limiting normalization.
The three standard types are the Gumbel distribution
the Fréchet distribution, with ,
and the negative Weibull distribution, with ,
They are max-stable distributions, as follows respectively by taking , , and . The necessary and sufficient classification is: is max-stable exactly when for , , and one of these three types . A positive affine map sends a max-stable distribution to another max-stable distribution.
For the requested threshold equivalence, set . By independence,
If , then and the expansion gives , since . Exponentiation proves the claimed limit. Conversely, if , then : any subsequence with would have its powers tending to zero. Taking logarithms now gives . Since , including the continuous extension at , it follows that . Hence
The exceedance count has the binomial distribution with parameters . For and each fixed nonnegative integer , its probability mass function satisfies
where and . Indeed , , and . If , the bound gives the degenerate Poisson distribution at zero. Summing the finitely many terms proves, in both cases,
This derives the needed instance of the Poisson limit theorem directly.
Finally fix with and put . The assumed limit for the normalized sample maximum and the proved equivalence give . The second-largest order statistic is at most exactly when at most one observation is strictly greater than . This event identity holds even if observations have ties. Apply the preceding Poisson limit theorem with to obtain
The endpoint corresponds to and is already included. This is the case of the exceedance count limit for extreme order statistics.
In the slotted ALOHA model, a station retains one packet until it successfully transmits. Conditional on , take fresh independent trials with a Bernoulli distribution of parameter for the backlogged stations. Let arrivals be independent across slots and independent of these trials, with Poisson distribution of mean . New arrivals join the next slot's backlog. An idle slot serves nobody, exactly one attempt serves one packet, and a collision serves nobody. Consequently the backlog changes by arrivals minus the success indicator.
Under these independence assumptions, is a time-homogeneous Markov chain: the feedback probabilities and the independent arrival law depend only on its current state. Merely specifying Poisson arrival marginals would not suffice. For example, take , , , and arrivals , , for independent random variables with Poisson distributions. Histories with and both have current backlog two, but the next backlog is respectively two and four. The usual model therefore includes fresh arrivals as an assumption.
Write and . Set for , and use their exact binomial distribution probabilities at . Away from the clipping boundary, the conditional drifts are
Near , the latter must use for each update , rather than itself.
When both coordinates are large with , the Poisson limit theorem approximates the attempt count, which has a binomial distribution, by a Poisson distribution of mean . Thus and . Rescaling state by and slot time by motivates the fluid approximation of slotted ALOHA
This is an interior fluid approximation of slotted ALOHA, not the exact conditional drift at a clipped boundary.
Here is one explicit set of sufficient conditions, independent of the arrival rate within the subcritical range:
Put . Then
Hence is negative below one and positive above one: the controller decreases the attempt denominator when offered contention is too small, and increases it when contention is too large.
Use the ratio time change for a homogeneous fluid model, . The equations become
For ,
since and . Also . Therefore is negative on , and by continuity on for some . Its value at zero is , so nonnegative backlog is preserved.
The ratio remains bounded by and eventually enters : on the compact interval , its derivative is bounded above by a strictly negative number. Once inside it cannot cross upward. The bounded ratio and smooth coefficients make the transformed equations exist for all . In this region , so decreases at least exponentially in . The original time satisfies and has a finite limit ; boundedness of gives as well. Thus every nonnegative interior fluid trajectory drains to the origin in finite fluid time. At the origin the ratio equation is undefined; the usual stopped fluid trajectory is held there afterward. A state with enters the interior under the continuous limiting boundary drift ; the same argument then applies.
This proof supplies sufficient conditions, not a characterization of every stabilizing triplet. It also does not assert the positive recurrent Markov chain property of the original stochastic chain solely from the heuristic ODE.
If , the ALOHA throughput bound gives
No choice of the three feedback increments can make this fluid backlog drain, since . For the displayed controller, the supercritical fluid trajectory is global and grows instead of draining. At the ray consists of stationary fluid states, explaining why the strict load inequality matters.
For fixed and sufficiently large ,
The finite product and last factor tend to one, while . Thus the Poisson limit theorem gives
These numbers form a probability mass function because they are nonnegative and their sum is , by the exponential series. They define the Poisson distribution of parameter . Only is needed to make the binomial parameter valid, which is sufficient for the limit.
Use the slotted ALOHA model with one outstanding packet per active station and a collision channel: a slot with exactly one attempt succeeds, whereas an idle slot or a slot with two or more attempts delivers nothing. Let be independent Poisson random variables of mean , independent also of the transmission choices. Arrivals join after the current slot's attempts. Given , the attempt count has a binomial distribution with parameters and . Consequently, for ,
For , set and . The equation adds arrivals and removes precisely one packet following a successful slot. The estimate controls the common transmission probability and updates using the idle, success, or collision feedback. The conditional distribution of the next state depends only on , so this pair is a Markov chain.
Away from the reflecting floor in , its one-step conditional expectations are
When and are large with , the Poisson limit theorem gives and . Treating these approximate drifts as derivatives motivates the fluid approximation of slotted ALOHA:
The floor at is negligible in this interior scaling. This calculation motivates the ordinary differential equations; it is not a proof of a stochastic fluid limit.
For , put
Then and . On the domain , introduce time by . Differentiating gives
where .
The function is strictly increasing from zero to infinity, and . To check the monotonicity explicitly,
Indeed, with ,
because its discriminant is ; and . Thus the scalar ordinary differential equation for points toward the unique value satisfying . Every trajectory with , has bounded and . Positivity of follows from .
If , then and . Thus decays exponentially for large , and
This is finite-time draining of a fluid model. The unmodified equations have undefined at the origin, so the statement that trajectories converge to the origin needs this endpoint interpretation; continuing with an absorbing zero trajectory is an additional fluid-model convention, not a solution of the displayed equations at the origin.
If , then and . Now grows exponentially, physical time tends to infinity, and
In particular the trajectory escapes to infinity. Even without the ratio analysis, , since .
For the actual Markov chain, it is essential not to replace the finite- success probability by : a single backlogged packet can succeed with probability one. Instead, for , maximizing over gives the ALOHA throughput bound
If , choose and so that for all , uniformly in . For sufficiently small ,
because its derivative at zero is . Independence of arrivals and attempts gives
whenever . For any integer , stop at the stopping time . The stopped process is a nonnegative supermartingale. The optional stopping theorem, first at bounded times and then by a limit, yields the exponential-supermartingale escape bound
From every state with , a Poisson distribution arrival burst has a uniformly positive probability of sending the next backlog to at least , for any fixed . From there the probability of never revisiting is at least . By the Markov property at successive visits, the probability of infinitely many visits to this strip is zero: each visit has a uniform positive chance of being the last. This holds for every integer , so
Therefore the Markov chain is transient for every finite choice of . The proof is uniform in and does not infer stochastic transience merely from the approximate ordinary differential equations.
Writing for team1_goal and for team1_xg, the first hypothesis concerns only the expected value:
The second specifies the entire conditional probability distribution:
with independence between matches as assumed in the question. The Poisson limit theorem, often called the law of small numbers in this setting, suggests a Poisson distribution when a goal count is the sum of many approximately independent rare scoring opportunities.
Let be independent variables and . Apply the tensorization of entropy to and then apply the stated Bernoulli log-Sobolev inequality in each coordinate. If , this gives
For each fixed , converges in distribution to , where . The Poisson limit theorem in fact gives convergence in total variation. The assumptions and make all displayed integrands bounded, so expectations and entropy pass to the limit. Since ,
Poisson distribution Created 2026-09-24 Updated 2026-10-14
For , a Poisson random variable has probability mass function
The exponential function series verifies normalization. At , define with certainty. The equality of mean and variance is a distinguishing model property, though it does not by itself characterize the law.
When counting events in a homogeneous Poisson process, distinguish its rate from the distribution parameter: an interval of duration has . A rate measured per minute times a duration measured in minutes gives a dimensionless expected count. This process assumes independent increments, a constant rate, and a negligible chance of multiple events relative to interval length in a sufficiently short interval. A rate alone does not establish a Poisson distribution. See Bertsekas and Tsitsiklis, chapter 6 summary, pages 46-47.
For a constructed arrival model with events per minute over minutes, . The mean and variance are both three, and
The probability of at least one event is . These results describe the specified process, not every arrival mechanism with the same average rate.
The Poisson limit theorem, often called the law of rare events, connects this law to the binomial distribution. Keep fixed and let for integers . For each fixed nonnegative integer and sufficiently large ,
There are more opportunities, each less likely to succeed, while the mean stays and the variance approaches . For , the probability of three successes is at , at , and at , approaching the Poisson distribution value .
Figure 1.
Binomial counts approach Poisson counts while their mean stays fixed at three
. Details.
The panels show the same count scale and the same Poisson distribution reference. At , is not especially small and the difference is visible. No universal trial-count cutoff guarantees an adequate approximation for every event.
The independence assumption matters even when every success is rare. For , if all indicator random variables equal one shared Bernoulli random variable of parameter , their sum is zero with probability and with probability . Its mean is three, yet its zero-count probability tends to one, not . Some dependent counts still admit approximation through the Stein-Chen method, but this requires additional control of dependence.
For historical orientation, the BnF records Poisson's original probability treatise, Recherches sur la probabilité des jugements, as published in Paris in 1837. This bibliographic fact alone does not establish when the modern distribution terminology arose. See BnF record of Poisson's 1837 treatise.
Poisson limit for occupancy fractions Created 2026-10-06 Updated 2026-11-05
In a uniform balls-in-bins allocation with and , a fixed bin count has convergence in distribution to a Poisson distribution variable of mean by the Poisson limit theorem. If counts bins containing exactly balls, symmetry and the linearity of expectation give for each fixed . In particular the expected empty fraction tends to , and the expected fraction with at least two balls tends to . These are statements about expectations; convergence of the random fractions requires an additional argument.
Poisson limit theorem Created 2026-09-24 Updated 2026-11-05
Let have the binomial distribution with trial count and success Probability . If , then
This is the Poisson limit theorem: many independent rare trials retain a finite total expected count. For a proof using characteristic functions,
The logarithmic expansion error vanishes because . The limit is the Poisson distribution characteristic function, so the Lévy continuity theorem proves convergence. The case is the degenerate count zero; it also follows directly from Markov's inequality.[ref]
For example, is approximated by , and its zero-count Probability is about , close to . The limiting mean is the limit of the product, not either individual parameter. independent unequal-probability Bernoulli trials have a related Poisson approximation when their largest success Probability vanishes and their total mean converges; dependence requires additional control, as in the Stein-Chen method.