Asymptotic distribution of a sample median Created 2026-09-24 Updated 2026-11-05
Let be independent and identically distributed random variables with unique median , and let their probability distribution have continuous probability density function with . For odd , the sample median satisfies
The influence function of the sample median has mean zero and variance : each of its two nonzero values occurs with probability . Its asymptotic linear representation, followed by the central limit theorem, gives the displayed limit.
For a standard normal distribution, , so the asymptotic variance of the unscaled sample median is . The sample mean instead has variance in that model. This quantifies an efficiency cost at the reference model without contradicting the sample median's bounded response to an outlier.
Asymptotic information loss from grouping Created 2026-09-24 Updated 2026-11-05
Suppose are independent and identically distributed with an exponential distribution of rate , but only the bin indices are observed, for known bin width . These are geometric random variables on the nonnegative integers, with
When , maximizing the grouped likelihood gives the rate maximum-likelihood estimator
If all bin indices are zero, the likelihood increases toward its supremum as , so there is no finite rate maximizer.
Using , the central limit theorem and delta method give
For the ungrouped observations, has limiting variance on this scale. The grouped limiting variance equals , which exceeds for positive . Thus grouping reduces asymptotic relative efficiency by the ratio . This ratio tends to one as : sufficiently fine bins lose little information, whereas a fixed bin width causes a genuine persistent precision loss. Setting recovers .
For a normal sample of variance , define . The exact chi-squared distribution identity and the central limit theorem for independent squared standard normal random variables imply
The delta method for , with derivative , therefore gives the displayed limit for . Exact independence of the normal sample mean and sample variance also makes independent of . The resulting joint limit supports combining location and scale estimates when a model relates their parameters.
For independent with replicates per group and , , let and . The likelihood equations give , , and when . Boundary totals give infinite logits. With fixed and , the central limit theorem and delta method give
The other groups do not increase information about the baseline logit when they have free offsets. For two groups, the information matrix for is with , whose inverse has first diagonal entry .
Binomial proportion Created 2026-10-14 Updated 2026-11-05
For with , the sample binomial proportion is . Its expectation is p and its variance is . The central limit theorem gives a normal approximation when both expected outcome counts are sufficiently large. These moments determine a standard error and model-based control limits.
Central limit theorem Created 2026-09-24 Updated 2026-10-14
Let be real independent and identically distributed random variables with expected value and finite, strictly positive variance . Write and for their sum and sample mean. The central limit theorem states
Precisely, for every real ,
where is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.
The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: . The division implements variance normalization: variance additivity for independent random variables gives , so every has expected value zero and variance one. For nonzero , an uncentered sum drifts; without rescaling its spread grows.
For a concrete example, let each have a Bernoulli distribution with success probability . Then has a binomial distribution, with expected value and variance . Its exact probability mass function and its normalized count are
Here is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.
Figure 1.
Exact standardized binomial distributions approach the standard normal curve
. For independent Bernoulli trials with , each blue bar has width equal to the normalized lattice spacing and area equal to the exact binomial distribution mass at its center. Heights are mass divided by spacing, so they can be compared with the orange probability density function of the standard normal distribution. All four normalized counts have expected value zero and variance one. The rectangles visualize discrete masses; they do not make the discrete probability distributions continuous. Details.
At , has expected value and standard deviation . The exact binomial distribution gives . A normal approximation with the continuity correction uses the interval to :
The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.
The strong law of large numbers and weak law of large numbers locate the average: , respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is , with standard error . This explains why four times as many independent observations halve the standard error. When is known, has approximately coverage for at large , giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.
A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when .[ref]
There are three useful limits to this picture:
A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.[ref]
A brief proof outline uses the characteristic function of . Its expected value is zero and its variance is one, so the finite second moment gives
The Taylor expansion of the complex exponential, with its remainder controlled by , and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of :
The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.
For an aperiodic positive Harris recurrent Markov chain with geometric ergodicity, an observable with stationary moment for some satisfies a central limit theorem. Its asymptotic variance is , with stationary covariances. Ergodicity alone does not guarantee this theorem.
Let be real independent and identically distributed random variables, with mean , variance and finite fourth central moment . For the sample variance , the displayed convergence in distribution holds.
Indeed . The central limit theorem for the sample mean implies that tends to zero in probability. Changing the divisor from to also has negligible effect at this scale, by the law of large numbers. Apply the central limit theorem to the independent variables , whose variance is , and use Slutsky's theorem. Standardization gives a standard normal distribution limit when . The familiar special case holds for the normal distribution because then .
Certain ruin with nonpositive loading and finite claim variance Created 2026-10-06 Updated 2026-10-07
For positive claims with finite variance and , the classical risk model has ruin probability one from any finite capital. At claim times, surplus increments are independent copies of with mean . Negative mean sends their partial sums to minus infinity by the strong law of large numbers. At zero mean, the increments have finite nonzero variance. For every fixed , the central limit theorem gives limiting probability of a partial sum below . The probability of unboundedness below is therefore at least ; as a tail event it has probability zero or one by the Kolmogorov zero-one law, and hence one.
Let be independent and identically distributed random variables taking and with equal probabilities, the Rademacher distribution. The two opposite complex exponentials give . Their expected value is zero and their variance is one. For , the characteristic functions of its sample mean and its variance normalization are respectively
For each fixed , the bases are positive for all sufficiently large . The Taylor expansion at zero then gives
Exponentiating proves the two displayed pointwise limits. The first limit is the characteristic function of the constant zero; the second is that of the standard normal distribution. The Lévy continuity theorem gives the corresponding convergence in distribution statements. Averaging has variance , whereas dividing the sum by preserves variance one. This sign calculation illustrates why the law of large numbers and the central limit theorem use different scales; it is not a proof for arbitrary probability distributions.
Debiased-Lasso asymptotic normality Created 2026-09-24 Updated 2026-11-05
In , put and choose a matrix from alone. The corrected Lasso estimate
has the exact decomposition
If , every remainder coordinate is bounded by . Thus a sufficiently accurate approximate inverse of a Gram matrix and a small coefficient L1 norm error make the remainder negligible.[ref]
Given , Gaussian noise makes the first term exactly normal with covariance matrix . For coordinate , let . If the remainder divided by tends to zero in probability, then converges to a standard normal distribution. Replacing requires a consistent estimator; non-Gaussian errors require a corresponding central limit theorem.
For independent Bernoulli random variables of fixed success probability , put and . The central limit theorem gives . Applying the delta method to the odds map , whose derivative is , gives
The ratio is undefined on the all-success event, whose probability is . Any finite extension there has the same limiting probability distribution, since that probability tends to zero. This is an asymptotic statement and does not assert an exact finite-sample variance of the unextended ratio. It is useful for uncertainty in estimated odds when the success probability stays away from one.
Edgeworth series Created 2026-10-05 Updated 2026-11-05
An Edgeworth expansion refines the central limit theorem by adding corrections determined by higher cumulants. Let be independent and identically distributed with mean zero, variance and skewness . For the standardized sum , the first density correction is
Here is the standard normal density. This is an asymptotic statement for a sequence, not a universal density formula for one arbitrary random variable. Sufficient smoothness and moment conditions are needed; one strong sufficient setting is that the common probability density function is a non-degenerate Schwartz function. In that setting the displayed remainder is valid for each fixed . Lattice-valued variables need additional corrections rather than this density formula.[ref]
To identify the coefficient, expand the log characteristic function of the standardized sum:
for fixed . Exponentiation and Fourier inversion turn into the Hermite polynomial times . The smoothness assumptions justify inversion beyond this formal calculation. Higher-order terms use further cumulants and Hermite polynomials.
The approximation improves central asymptotics, but it need not be a nonnegative normalized density after truncation. In particular the cubic correction can become negative in a far tail, where a fixed-point expansion does not provide a small relative error. Inverting a distribution-function Edgeworth expansion produces the Cornish-Fisher expansion for fixed quantiles.
For an independent statistical sample from the beta distribution with parameters , , the probability density function is on . Its raw moments are for , by direct integration. Consequently the method of moments gives , whereas the maximum-likelihood estimator is .
The second formula follows by setting the score function to zero; its derivative is negative. Also has an exponential distribution of rate , since . The central limit theorem and delta method give
Thus the asymptotic relative efficiency of relative to is . The transformed sum has a gamma distribution of shape and rate , which also gives for and for .
Exponential-rate maximum-likelihood estimator Created 2026-10-03 Updated 2026-10-05
For independent observations from the exponential distribution with rate , the log-likelihood is , so
The central limit theorem and delta method give
Extreme value theory Created 2026-10-05 Updated 2026-10-14
Extreme value theory studies limiting probability distributions of normalized sample maxima and related extremes. The standard limits for maxima include the Fréchet distribution, negative Weibull distribution, and Gumbel distribution.
The question is often practical: if a batch finishes only when its slowest task finishes, its completion time is a sample maximum. A typical individual delay or a sample mean does not answer that question. The central limit theorem describes suitably normalized sums and sample means under its assumptions; it does not give a limit law for sample maxima. Extreme value theory instead describes how the largest observation changes as the number of opportunities for a large observation grows.
For independent and identically distributed random variables with distribution function , define the sample maximum
The event means that every observation is at most , so independence gives
Extreme value theory asks whether there are constants and for which has convergence in distribution to a non-degenerate probability distribution. Equivalently, must tend to a limiting distribution function at its continuity points. The extremal types theorem says that any such limit belongs, up to a change of location and positive scale, to the Gumbel distribution, Fréchet distribution, or negative Weibull distribution. It does not guarantee that a limit exists for every . [ref].
The maximum of independent exponential random variables gives a particularly transparent worked example: its center grows as , and its fluctuations have a Gumbel distribution limit on the scale . Here and below denotes the natural logarithm. This normalization depends on the exponential distribution. Other survival functions can lead to the Fréchet distribution or negative Weibull distribution, with different constants ; the Gumbel distribution and the exponential example's normalization are not universal.
Independence is essential to the product calculation. If instead for one rate- exponential random variable, then for every . Its distribution function stays for and zero below zero, and its median stays . Perfect dependence creates no logarithmic growth: repeating the same random variable supplies no new chance of a larger value.
A historical starting point is R. A. Fisher and L. H. C. Tippett's 1928 paper on limiting laws of the largest or smallest sample member, published in volume 24, issue 2, pages 180–190 of the Proceedings of the Cambridge Philosophical Society. Its abstract studies the possible limiting forms and emphasizes that approach to a limit can be slow. [ref].
FPCA mean test Created 2026-09-24 Updated 2026-11-05
An FPCA mean test uses functional principal component analysis to test a zero expectation in a Hilbert space. For independent and identically distributed observations with finite second moment, let be the eigenvalues and orthonormal eigenvectors of the covariance operator. Choose a fixed positive integer and, as a sufficient separation assumption, require and . With consistent estimators of these eigenvalues and eigenvectors, the statistic is
Under the null hypothesis , the central limit theorem gives joint convergence of to independent standard normal random variables. Statistical consistency and Slutsky's theorem justify replacing the population eigenvectors and eigenvalues by their estimates. Consequently . The fixed-dimensional projected two-sample version has the same calibration.[ref].
The positivity condition prevents division by zero; the spectral gap makes the estimated directions stable. Repeated eigenvalues can be handled through an entire retained eigenspace, provided its boundary remains separated. A choice growing with needs additional approximation theory. A fixed can miss a nonzero expectation lying entirely in the omitted orthogonal directions, so the test's simplicity comes with a choice about which deviations matter.
Functional statistic 2026-10-14
A functional statistic evaluates a statistical functional at the empirical distribution . The map acts on distributions, whereas is a statistic. If has an asymptotic linear representation, then
A square-integrable influence function with mean zero consequently gives asymptotic variance by the central limit theorem.
For independent identically distributed mean-zero variables with finite positive variance, the central limit theorem gives a positive eventual lower bound for for every finite . The decreasing events have probabilities bounded below, so continuity from above of a measure gives positive probability that the limsup is at least . The limsup itself is unchanged by removing any finite initial sum, hence is tail measurable. Kolmogorov zero-one law makes each event “limsup at least ” certain; intersect over integer . The result is false for zero variance. One should apply the zero-one law to the limsup, rather than assume the exact threshold infinitely-often event is unaffected by a vanishing change.