Thermal fluctuation Created 2026-10-13 Updated 2026-11-05
A thermal fluctuation is equilibrium variation of a random variable representing an observable. In a classical canonical ensemble with , assume and the state space measure are independent of , and fix temperature . Differentiating the normalized Boltzmann distribution gives
Thus the variance measures response to a conjugate field. For a two-state observable with energies , and , verifying the identity directly.
The same simple formula holds in a finite quantum system when the commutator of and vanishes. For noncommuting quantum observables, differentiation of the operator exponential produces an imaginary-time-ordered correlation function response rather than automatically . In a classical field theory, the spatial two-point analogue uses the connected correlation function.
Total correlation Created 2026-09-28 Updated 2026-10-14
Total correlation measures how much dependence is lost when a joint probability distribution is replaced by the product of its marginal probability distributions. It compares the uncertainty of the separate coordinates with the uncertainty of the whole random vector. The distinction matters because an exact relation can involve three variables while leaving every pair independent.
For random variables with finite alphabets, define
Here is information entropy, the entropy of the full random vector is joint entropy, and is relative entropy. Base-two logarithms give bits; natural logarithms give nats. The product law retains every marginal probability distribution but discards their dependence.[ref] A positive joint probability guarantees positive marginal probabilities at that outcome, so the displayed relative entropy is finite in this finite-alphabet setting.
For two variables, total correlation is exactly mutual information. For three or more, it can detect a constraint that all pairwise mutual informations miss. Relative entropy nonnegativity gives , with equality exactly when the coordinates are jointly independent random variables.
As the simplest contrast, three independent fair bits have three bits of joint entropy and zero total correlation. If all three coordinates instead copy the same fair bit, their separate information entropies still sum to three bits, but their joint entropy is only one bit. Their total correlation is therefore two bits. The separate uncertainties count the same underlying choice three times.
Suppose one observation comes from either or , each with prior probability . A decision rule that chooses on an event and otherwise has success probability
On a finite set, choosing therefore gives . This is the equal-prior version of minimum sum of errors in a simple hypothesis test; allowing a statistical test to use an auxiliary random variable cannot improve it, because for each outcome the better of the two decisions already maximizes its contribution.
For the four-outcome example, choose after and after or , with either decision allowed after . Choosing after yields . The best success probability is 65%, with a 35% error probability. A smaller total variation distance means the two sources are harder to distinguish from one observation. Unequal prior probabilities, unequal costs, or multiple observations change this particular decision problem; the figure is not a universal bound for every experiment.
Truncated distribution Created 2026-09-24 Updated 2026-11-05
A truncated distribution is the conditional probability distribution obtained by restricting a random variable to a specified event with positive probability. If its original law is , the truncated law is
The denominator renormalizes the retained probability mass to one. If has a probability density function , the truncated density is ; the same restriction and normalization apply to a probability mass function.
For example, truncate a uniform distribution on to values at least six. The retained event has probability , so the new density is on , and its mean is eight. This is the law obtained by repeatedly sampling from the original law and rejecting values outside the retained set; rejection sampling becomes inefficient when the retained event is rare.
A zero-probability restriction cannot be normalized by the displayed formula. Conditioning a continuous random variable to equal a specified value requires an additional conditional-law construction or limiting convention, rather than division by zero.
Truncated first moment bound Created 2026-10-06 Updated 2026-11-05
For a nonnegative random variable with finite mean and , the part of the expectation on is at most . Subtracting gives . If the second moment is positive and finite, applying the Cauchy-Schwarz inequality to and the indicator variable gives the Paley-Zygmund inequality .
Stipulate an original Gaussian mixture distribution with mixture weights , , component means , , and component variances both equal to one. These are chosen statistical parameters, with no observed sample or fitting. If is the standard normal distribution probability density function, then
In the figure, the two colored curves are weighted probability density functions, with areas and . Adding their heights at every gives the black curve. Neither colored curve stops at a dividing line.
Figure 1.
Stipulated two-Gaussian mixture: weighted component densities and their sum
. Original analytic curves for component means and , component variances one, and mixture weights and . No observed points, histogram, or fitted parameters are used. Details.
The law of total expectation and law of total variance give the total mean and variance:
All of the extra variance comes from variation between component means. Equivalently, the second moment is , so the variance is .
A mixture distribution is different from a deterministic linear combination of random variables. If independent and are drawn and , then has a normal distribution with the same mean but variance , rather than . Every draw of uses both random variables. A draw from the mixture distribution instead uses , where the mixture allocation indicators are independent of and select exactly one of them. The expression for adds densities, not simultaneous random outcomes.
Uncorrelated random variables Created 2026-09-24 Updated 2026-10-14
For an explicit counterexample, let take , , with equal probabilities, and let . Then , so . Nevertheless, is completely determined by : , while . Thus these are uncorrelated random variables with statistical dependence. In this example, opposite contributions from the odd function cancel; the even function still retains information about . The last panel of the covariance diagram shows the relationship.
There are useful special cases. Uncorrelated jointly normal variables are independent, and the Bernoulli independence from zero covariance result supplies another setting where the converse holds. Their distribution assumptions do the extra work; zero covariance alone does not.
Uniform consistency of an empirical moment-generating function Created 2026-10-07 Updated 2026-11-05
Let be independent and identically distributed nonnegative random variables. Fix with , and define
Then the empirical moment-generating function is uniformly strongly consistent on this interval:
The dominated convergence theorem, with envelope , makes continuous on . Both and every are nondecreasing because the observations are nonnegative. Given , choose a finite grid whose successive increments are at most . The strong law of large numbers holds simultaneously at its finitely many endpoints. Squeezing between adjacent endpoints gives
Use a countable sequence to conclude. continuity of the sample and population functions also makes the supremum measurable by reducing it to rational parameter values. This is the finite-bracketing mechanism behind the uniform strong law from finite L1 bracketing. The iid sampling assumption cannot simply be omitted: repeatedly observing the same nonconstant random variable would not create averaging.
Uniform integrability Created 2026-09-24 Updated 2026-11-05
A family of integrable random variables is uniformly integrable when
Almost-sure or probabilistic convergence together with Uniform integrability upgrades to convergence.
Varentropy Created 2026-09-24 Updated 2026-11-05
The varentropy of a discrete random variable is the variance of its information content:
It measures variation in surprise across outcomes, while Shannon entropy is the expected value of that surprise. Assuming finite Shannon entropy, the variance may still be infinite; finite varentropy requires an integrable squared logarithm.
For a Bernoulli random variable with success probability ,
Indeed the information content takes values and , and the variance of a two-valued random variable is times the squared difference. At , every outcome supplies one bit, so Shannon entropy is maximal but varentropy is zero. For independent and identically distributed random variables, the information content of a length- word is a sum, giving variance when finite.
Variance Created 2026-09-24 Updated 2026-10-14
The variance measures the mean squared distance of a random variable from its expected value. For a real square-integrable random variable ,
The square prevents positive and negative deviations from cancelling. It also gives the variance squared units: if measures a length in metres, its variance has units of square metres. Its nonnegative square root is the standard deviation, which has the same units as .
For example, let take , and with equal probabilities. Its expected value is zero, but its variance is , and its standard deviation is . The constant random variable zero has the same expected value and zero variance: a mean alone does not describe the spread. The variance is zero exactly when almost surely.
The variance is useful because squared deviations interact neatly with linear combinations. The covariance supplies the cross terms when combining different random variables, and a covariance matrix organizes those terms for many variables. The characterization of variance as the minimum mean squared error of a constant also explains why the expected value is the best constant prediction under squared error.
Variance additivity for independent random variables Created 2026-09-24 Updated 2026-11-05
For independent square-integrable random variables,
Variance as the minimum mean squared error of a constant Created 2026-10-03 Updated 2026-11-05
For a square-integrable real random variable ,
Thus the unique best constant predictor is , and the minimum mean squared error is .
Variance normalization Created 2026-09-24 Updated 2026-11-05
Variance normalization rescales a centered sum to have variance one. Let and assume . Then
For independent variables with finite variances, . With dependence it also contains . For example, if every equals the same random variable of variance , the sum's variance is , not .
This is the natural scale for central limit theorems, but variance one alone does not imply a Normal distribution or convergence to one. Appropriate assumptions, such as a Lindeberg condition for a suitable independent triangular array, control the shape of the normalized sum. Centering, choosing the correct variance and proving a distributional limit are distinct steps.
Variance of a sample mean Created 2026-09-24 Updated 2026-10-14
Let be an IID sample of real-valued random variables with finite expected value and finite variance , where . Its sample mean satisfies
Linearity of expectation gives the first identity. The variance of a sum and vanishing cross covariances give ; scaling by scales the variance by . Pairwise independent random variables with the same expected value and variance suffice for these identities, even without full independence. Grinstead and Snell state the corresponding finite-variance independent-trials result.[ref]
The standard deviation of the sample mean is . Four times as many observations halve this scale; they do not divide it by four. In the fair Bernoulli distribution example, , so sample sizes and give standard deviations and , respectively. The common-coin averaging example shows why merely repeating observations with a common marginal law does not justify this calculation.
Chebyshev's inequality turns the variance identity into a probability bound: for ,
This proves convergence in probability of the sample mean to in this finite-variance setting, a form of the weak law of large numbers. It is an upper bound, so values above one convey no improvement over the universal probability bound one. Cauchy sample mean stability lies outside these hypotheses.
Variance of a sum Created 2026-09-29 Updated 2026-11-05
For square-integrable random variables,
Waiting-time distribution Created 2026-10-25 Updated 2026-11-05
A waiting-time distribution is the probability distribution of a nonnegative random duration until a specified event. It can be discrete, absolutely continuous, or mixed. Its survival function is and its cumulative distribution function is .
If the finite-time law is absolutely continuous, the probability density function is almost everywhere. Mere continuity of the cumulative distribution function is insufficient for this density representation. At times with positive survival, the hazard function is ; it measures the instantaneous event rate conditional on having waited that long.
Constant hazard with gives , the exponential distribution. Waiting a discrete number of independent Bernoulli trials instead gives a geometric distribution. A fixed deterministic wait has a point mass at its prescribed duration rather than an ordinary density.
Wasserstein distance Created 2026-09-24 Updated 2026-11-05
The first Wasserstein distance measures the least expected separation achievable when two random variables are coupled with specified marginal probability measures. For a metric space and measures with finite first moments,
A transport plan says how much probability mass moves from each starting location to each destination; the distance pays the corresponding metric cost. For point masses on the real line, , because there is only one coupling. In contrast, their Total variation distance is one whenever . Wasserstein distance therefore records that nearby displacements are small even when the two laws give mass to disjoint sets.
Every coupling gives an upper bound by its expected separation. Conversely, the Kantorovich–Rubinstein theorem gives lower bounds from differences of expectations of Lipschitz functions. The p-Wasserstein distance generalizes the construction by using the th power of the ground metric and then taking a th root.
Wigner matrix Created 2026-10-07 Updated 2026-11-05
A real Wigner matrix is a real symmetric matrix whose upper off-diagonal entries are independent centered copies of a fixed random variable, scaled to have variance of order . Diagonal conventions may vary and should be specified separately. A common version has variance-one unscaled off-diagonal entries and either zero diagonal entries or independent diagonal entries with a bounded second moment.