Strong law of large numbers Created 2026-09-24 Updated 2026-10-14
Let be real independent and identically distributed random variables with , and let be their expected value. Their sample mean satisfies
This is almost sure convergence: with probability one, an entire infinite sequence of observations has averages that eventually stay within every prescribed positive tolerance of . The variance need not be finite. The strong law of large numbers implies the weak law of large numbers, which asserts convergence in probability at each increasing sample size.
For independent trials with a Bernoulli distribution of parameter , is one for success and zero for failure. Then is the observed success proportion, and the strong law of large numbers says almost surely. This turns a theoretical probability into a quantity one can estimate by repetition. It does not say that each new trial improves the estimate: a long string of failures can move the proportion away from before later observations bring it back.
Figure 1.
Bernoulli averages approach their mean: sample paths and exact error probabilities
. The left panel shows four reproducibly simulated sequences of independent Bernoulli trials with success probability . The right panel calculates from the exact binomial distribution and compares it with the bound from the Chebyshev inequality. Small irregularities reflect integer-valued counts. The paths illustrate almost sure convergence; a finite simulation cannot prove it. Details.
The strong law of large numbers gives the limiting location of the sample mean, while the central limit theorem describes its fluctuations when the variance is finite and positive. The two assertions can hold together:
The first error shrinks to zero; the second has been enlarged enough to retain a nontrivial probability distribution. The second assertion is convergence in distribution, not a claim that the enlarged error along one sequence approaches a particular normal random variable.
The assumptions make a practical difference. If every observation repeats the same nonconstant Bernoulli random variable, , then for all : identical probability distributions alone do not make repeated observations informative. For independent observations with a Standard Cauchy distribution, the Cauchy sample mean has an unchanged distribution, so it does not concentrate around a constant. Its expected value is undefined, and ; neither example contradicts the stated strong law of large numbers.
A brief proof outline in the case of finite variance uses the Kolmogorov maximal inequality for the centered partial sums. The probability that their maximum up to exceeds is at most . These bounds are summable, so the Borel-Cantelli lemma controls all sufficiently large such blocks and hence every intermediate sample size. The general case requires truncation to control large observations; the argument using finite variance alone does not prove that full version.
Studentization of the generalized covariance measure statistic Created 2026-09-28 Updated 2026-11-05
Suppose the scaled mean residual product in the generalized covariance measure statistic converges in distribution to , with , and its empirical variance converges in probability to . Then the Slutsky theorem gives . It is the square root of the variance estimate that belongs in the denominator. Under the null and suitable regression conditions, for the two true regression residuals. This is a calibration argument; it does not remove the conditions needed for the numerator limit or variance consistency.[ref]
Studentized sample mean Created 2026-10-25 Updated 2026-11-05
For independent and identically distributed observations with finite positive variance, the central limit theorem, consistency of the sample variance and Slutsky's theorem give . Under an independent normal distribution sample it instead has the exact finite-sample Student t-distribution with degrees of freedom, as described by the studentized normal sample mean. The asymptotic statement does not require normal individual observations.
Student's t-distribution Created 2026-09-29 Updated 2026-10-20
The Student's t-distribution describes a standard normal random variable divided by an independent estimate of its scale. For any real , let have the standard normal distribution and let have the chi-squared distribution, with and independent. Then
The parameter is called the number of statistical degrees of freedom. It need not be an integer: the chi-squared distribution is a gamma distribution with shape and rate . The independence assumption is essential; specifying only the two marginal distributions does not determine the probability distribution of their ratio.
Its probability density function on the whole real line is
where is the gamma function. Every member is a symmetric distribution about zero. At it is the Standard Cauchy distribution; as it approaches the standard normal distribution. The essential shape change is the tail: at any fixed finite , decays like a constant times , much more slowly than the standard normal density's exponential decay. Thus it is a heavy-tailed distribution. [ref]
The left panel shows the centers; the right panel uses a logarithmic scale to reveal the tails that a linear plot hides. All four curves have location zero and scale one. They do not have the same variance: the curve has no finite mean or variance, whereas has variance 3 and has variance .
More generally, the absolute moment for is finite exactly when , as follows by comparing the tail integrand with . Consequently
For there is no finite mean; symmetry alone cannot supply one. For the mean exists but the variance is infinite. For even the centered definition of variance is unavailable because the mean does not exist.
A shifted and scaled version , , has probability density function . The scale is not its standard deviation: for ,
For example, and scale give standard deviation , rather than 2. The distinction matters when using a Student's t-distribution to model data with occasional extreme values, as well as when interpreting a Student t confidence interval.
Student's t-test Created 2026-09-29 Updated 2026-11-05
Student's t-test calibrates a standardized mean contrast using a Student's t-distribution under a specified sampling model. For independent with and unknown positive variance, the one-sample statistic is
Under the null hypothesis , exactly.
The exact law uses two facts: is standard normal, and independently of . Hence , the defining construction of the Student's t-distribution. A normally distributed numerator divided by an arbitrary estimated standard error need not have this law.
For a two-sided level- test, reject when ; a directional alternative uses the corresponding one-sided quantile. The paired t-test applies the one-sample construction to independent pairs' differences. The exact pooled two-sample t statistic uses independent normal samples with a common variance; Welch's t-test instead estimates unequal variances and uses an approximate degrees-of-freedom calibration. The selected model and statistical degrees of freedom determine which reference distribution is justified.
Student t confidence interval Created 2026-09-24 Updated 2026-11-05
For a normal sample with unknown mean , positive variance and , let . Then
and an exact confidence interval for is .
The proof uses independence of the normal sample mean and sample variance: is standard normal, independently of . Dividing produces Student's t-distribution. The random scale increases uncertainty relative to a known-variance normal interval. Non-normal samples need an approximation argument; an independent scale estimate alone does not make an arbitrary statistic exactly Student t.
A Student t random-effect model assigns Student's t-distributions to exchangeable study effects, allowing heavier tails than a normal distribution hierarchy. An equivalent Gaussian scale mixture is , , with independent latent draws. For the variance is , so is a scale rather than a standard deviation. Small latent precisions weaken shrinkage for atypical studies while retaining partial pooling for the rest.
Sub-Gamma random variable in the right tail Created 2026-09-24 Updated 2026-11-05
A centered random variable is sub-Gamma in the right tail with variance parameter and scale parameter when
for .
A sub-Gaussian deviation bound controls tails through a Gaussian-shaped exponential. If a centered random variable satisfies for every real , with , then for ,
Indeed, exponential Markov's inequality gives ; minimizing at and applying the argument to proves the result.
Independent centered observations with a common variance proxy therefore satisfy . The proxy need not equal the actual variance, and dependence requires a separate argument rather than automatic multiplication of moment-generating functions.
Sub-Gaussian distribution Created 2026-10-13 Updated 2026-11-05
A real centered random variable is sub-Gaussian random variable with parameter if its moment-generating function satisfies
The parameter is an upper bound on fluctuation size, not necessarily the standard deviation. For and , exponential Markov inequality gives . Choosing and applying the same argument to yields the Chernoff bound
A centered normal distribution of variance attains equality in the moment-generating-function condition. A Rademacher distribution also satisfies it with , since . Products of moment-generating functions show that a sum of independent centered sub-Gaussian random variables with parameters is sub-Gaussian with parameter . Hence the sample mean of independent variables with a common parameter has tail bound . This gives normal-like concentration without assuming a normal distribution. For an uncentered variable, the convention is to apply the condition to .
Sub-Gaussian random vector Created 2026-10-06 Updated 2026-11-05
A centered random vector is sub-Gaussian with variance proxy when the displayed inequality holds for every deterministic vector . Equivalently every scalar inner product is a sub-Gaussian random variable with proxy . independence of coordinates is not required. For vectors with Euclidean norms at most one, the union bound gives with probability at least . This assumption alone is weaker than concentration for all Lipschitz functions.
Submartingale Created 2026-10-03 Updated 2026-11-05
A submartingale is an integrable adapted process satisfying almost surely. Its conditional tendency is upward; an individual sample path can still fall. If is a mean-zero random walk with step variance , then is a submartingale because . More generally a convex function of a martingale is a submartingale when integrable, by conditional Jensen inequality. Negating a submartingale gives a supermartingale.
Sub-Poisson random variable in the right tail Created 2026-09-24 Updated 2026-11-05
A centered random variable is sub-Poisson in the right tail with variance parameter when
for every .
Sub-step estimation with uniform dither Created 2026-10-20 Updated 2026-11-05
Let a constant input be with , between consecutive quantization levels and . Add independent centered uniform dither before nearest-level rounding, and assume that neither code clips.
The upper code is selected exactly when . This subinterval occupies fraction of the uniform distribution's support, so with a Bernoulli random variable of parameter . Thus and . Averaging independent conversions gives
For example, an input one quarter of a step above the lower code produces the upper code one quarter of the time. Its averaged code can estimate a sub-step voltage, with a finite remaining variance. Centering, independence and absence of clipping are assumptions of this calculation; a shifted dither distribution can bias the result.
Symmetric stable distribution Created 2026-10-14 Updated 2026-11-05
A symmetric stable distribution has this characteristic function, with and location . At location zero it is a strictly stable distribution: weighted sums of independent copies have scale factor . The index need not permit a finite mean or variance. The scale in this convention is , so the coefficient of and the scale parameter should not be confused.
Tagged-label Rayleigh quotient for adjacent transpositions Created 2026-10-06 Updated 2026-11-05
For the random adjacent transposition shuffle on with , the position of one label is uniform on in equilibrium, so . Each adjacent transposition moves the label with Probability , giving . The Poincare inequality for a reversible Markov chain variational formula therefore gives . In particular the continuous uniform distribution on has variance , not .
The tail assumptions needed for conservative Wald coverage concern the distribution of the standardized estimation error and the realized scale of its interval. An expected-value bound on a conservative variance estimator does not establish either property. The discrete and rare-large-variance counterexamples in conservative variance estimation does not guarantee Wald interval coverage isolate those failures. NIST similarly makes a normal-model qualification before interpreting an expanded uncertainty as approximately 95 percent coverage.[ref]
A precise sufficient asymptotic argument is the following. Let , take , and suppose
The first condition includes centring at the true parameter; asymptotically material bias would change it. The second condition is convergence in probability of a variance ratio, substantially more than an expectation inequality. By Slutsky's theorem, the Wald confidence interval with normal critical value then has limiting coverage probability
The limiting normal distribution has no mass on the interval boundary, which justifies passing to that probability. This is asymptotic conservative Wald coverage under variance-ratio convergence, not a finite-sample guarantee.
For an original exactly normal comparison, let a sample mean have true variance and use the deterministic scale . A nominal 95 percent Wald confidence interval uses a radius times the usual one and has exact coverage probability . The calculation uses an actual normal sampling law and a fixed ratio; it cannot be inferred from alone. The normal cdf and inverse-cdf implementations provide the numerical evaluation.[ref]
A samplewise variance upper bound supplies a separate finite-sample route when the centre is exactly normal, because each reported interval contains the known-variance normal interval. Without that shape assumption, use a justified tail inequality as in confidence bounds from a samplewise variance bound.
Tail-model dependence of inferred density uncertainty is the sensitivity of a density estimate's distribution to the assumed probability of very small positive thicknesses. For with fixed , a tiny thickness has disproportionate influence through . The main body of a thickness histogram can look stable while the inferred mean or variance remains model-dependent.
A gamma distribution and a lognormal distribution provide useful explicit models. NIST specifies their density functions and parameter conventions.[ref] At mean thickness and coefficient of variation , the gamma model has shape and gives
The lognormal model gives . The two agree closely for small , but that agreement concerns those model families, not every positive distribution with a similar spread.
An original rare-event example makes the limitation explicit. Let thickness be micrometres with probability and micrometres otherwise. Mean thickness is exactly 10 micrometres and its standard deviation is approximately 0.10 micrometres, about 1% of the mean. Nevertheless,
nearly twice the reciprocal of mean thickness. The rare component supplies about half the inverse moment. These are deliberately constructed model values, not measured polymer-layer dimensions.
Rare-thickness leverage on inverse moments can escape an apparently reassuring sample. With 1,000 independent thickness observations, the chance of seeing none of that rare component is . A finite simulation or a smooth fit to common observations need not validate the unobserved lower tail.
A useful experiment investigates whether dimensional resolution, specimen geometry or physical constraints rule out such near-zero layers. Report sensitivity to defensible alternative lower-tail models or measured lower bounds. Choosing a positive distribution guarantees positive thickness, but does not establish that its inverse moments describe the measurement process.
Figure 1.
An original two-point thickness model: a component appearing in only one of ten thousand observations supplies about half the expected reciprocal thickness, and therefore half the expected inferred density at fixed mass per area.
Details.
Taxi distance convolution on a rectangular road cycle Created 2026-10-14 Updated 2026-11-05
For a taxi starting at any vertex of a rectangle with side lengths , a uniform pickup vertex gives approach-distance law uniform on . Conditional on that pickup, a uniform distinct destination gives passenger-distance law uniform on , independent of the pickup vertex. Hence the two distance variables are independent despite sharing that vertex. Their convolution has twelve equally likely pairs. Its mean is and its variance is . Repeated numerical sums combine their multiplicities when some distances coincide.