Two-step prediction for an MA(1) process 2026-10-07
In a centered MA(1) process, is uncorrelated with every observation through time . Its best linear predictor from that information is therefore zero, with error variance . One-step innovation formulas using cannot be substituted when that observation is unavailable.
Two-track thermal-wind covariance Created 2026-10-25 Updated 2026-11-05
For a steady density field sampled by two ships at angles and zero, write . Independent measurement errors of variance giveNearly parallel tracks poorly constrain and induce positive component covariance for . Orthogonal tracks remove this covariance and give equal component precision under the stated independent-noise model.
Typical-set concentration for a biased coin 2026-10-14
The count has binomial distribution with mean and variance . The information content per symbol is an affine function of , so its variance isChebyshev's inequality therefore gives the finite-sample lower boundThe bound is often loose, but it proves that every fixed positive tolerance eventually captures nearly all the probability. Direct summation of the binomial distribution's probability masses gives . A finite sample does not need to have a nearly full typical set: the 40-symbol example above captures only about .
Longer biased-coin messages concentrate in typical sets with controlled size
. The left plot compares exact typical set probabilities with the lower bound from Chebyshev's inequality. The right shows and its finite-sample bounds. The tolerance stays fixed; the curve is not asserted to converge to exactly . All points are exact finite-model calculations. Details. Unbalanced sampling-uncertainty design 2026-11-05
An unbalanced sampling-uncertainty design estimates collection and analytical contributions while performing different numbers of analyses on the collected samples. One useful duplicate design collects two samples independently at each target, analyses one twice and the other once. The Eurachem guide presents this three-result arrangement as a less costly alternative to analysing both samples twice.[ref]
An original variance calculation shows why both components remain identifiable. At target , letHere labels independently collected samples; only has analytical replicates . Assume centered independent random effects, common variance components and the same analytical precision for each result. Define within-target contrastsThe target mean cancels. The first contrast also cancels the sample effect; the second compares independent collections with unequal analytical averaging. Consequently,The coefficient matters: one sample mean contains half the analytical variance, while the other result contains all of it. Applying the balanced-design coefficient would misallocate uncertainty.
For stipulated contrast variances and in squared concentration units, the corresponding method-of-moments estimates areA single analysis of a single collected sample then has modeled standard uncertainty in concentration units. These are invented values illustrating the model, not measurements from the guide.
Across eight targets, this design requires sixteen collections and twenty-four analyses, compared with thirty-two analyses for two analytical replicates on each collection. The saving is in analytical runs; equal cost does not imply equal statistical precision. Repeat the nested design across targets to estimate the contrast variances. One triple of results does not reliably estimate the collection variance, and duplicate collection by one unchanged protocol does not reveal a bias shared by every sample. A negative estimated variance component signals finite-data or model issues requiring an appropriate estimation method, rather than a physically negative uncertainty.
For independent with exponentially distributed at known rate , the joint likelihood factors through the sample minimum . Its excess over the endpoint is exponential with rate , because . Thus the displayed estimator is an unbiased estimator, with variance . Even if , the estimator can be negative; truncation at zero introduces bias.
Unbiased likelihood estimator 2026-10-06
An unbiased likelihood estimator has expectation equal to the likelihood function at each parameter. A pseudo-marginal algorithm requires this estimator to be nonnegative. Importance sampling provides for independent random variables , provided the proposal covers the target support. Relative variance affects the mixing time even though unbiasedness guarantees the correct marginal target.
Unbiased stochastic gradient Created 2026-09-29 Updated 2026-11-05
A random vector is an unbiased stochastic gradient at an iterate ifFor an adaptive iteration it is often useful to require the stronger historical condition , where contains all information available before drawing the new sample. This states explicitly which information is held fixed in the conditional expectation.
For , a conditionally uniform index gives . If index instead has positive sampling probability , then is the correctly weighted estimate. Omitting these factors generally changes its expectation.
Uncertainty in estimated inverse-variance weights arises when the variances used in inverse-variance weighting are themselves estimated. A very small estimated variance can give one observation or group a large random weight. The familiar variance formula for fixed, known weights cannot simply be reused with estimated variances treated as exact. NIST specifically warns that weights estimated from few replicates can substantially and unpredictably affect a weighted least squares analysis.[ref]
Suppose independent groups contain independent normal random variables with common mean and group variance . Let and be the group sample mean and unbiased sample variance, and formFor normal samples, each sample mean is independent of its sample variance. Conditional on the estimated weights, the estimator remains unbiased and hasThus the law of total variance gives unconditional variance equal to the expectation of this expression. The true still appear; replacing them by the same random does not establish the actual conditional variance. Outside the independence assumptions, even conditional unbiasedness can fail.
An original two-group example takes and true , so each group mean has variance . If observed sample variances are and , the estimated normalized weights are and . For group means and , the combined estimate is . The plug-in variance isbut the true conditional variance under the stipulated model isThe corresponding standard deviations are about and . Equal weights would have variance . An accidentally small estimated variance has both overstated precision and moved the combination away from the optimal equal weights in this example.
Reciprocal moments of an estimated variance explain the unstable precision estimate. For a normal sample with , has a chi-squared distribution. Integrating its density givesConsequently for , while the second moment requires . The density and its gamma-distribution parameterization are documented in the primary SciPy implementation notes.[ref] With , the inverse sample variance has a finite mean twice the true precision and an infinite variance; the normalized weights remain bounded, so this does not imply their variance is infinite.
More replication or a justified smooth variance model can stabilize weights. NIST describes direct replicate-based weight estimation and variance-function modeling as distinct approaches.[ref]
Weighted Least Squares by NIST. e-Handbook §4.4.5.2, Direct Estimation of Weights and Estimating Weights with a Variance Function
A proposed validation repeats the complete weight-estimation and combination procedure on simulated or replicated groups, checking actual interval coverage as well as point estimates. A bootstrap must reproduce the weight-estimation stage; holding the fitted weights fixed tests a different procedure. Uncertainty of a gravimetric dilution 2026-11-05
The uncertainty of a gravimetric dilution describes how uncertain mass measurements propagate into the retained stock fraction . Write , where is stock mass and is added diluent mass. The first-order propagation of uncertainty isThe negative covariance coefficient follows from the opposite signs of and . This is the covariance form of the uncertainty law applied to this specific measurement model.[ref]
For an original sequential-weighing example, let the empty vessel reading be , the reading after adding stock , and the final reading . Then , and . Suppose the three readings have independent standard uncertainties of , with other effects neglected. Both net masses have variance , butThus , a relative standard uncertainty of about . Omitting this shared-reading covariance would give and underestimate the uncertainty. Equivalently, propagate the three independent readings directly through .
A common multiplicative balance scale error instead cancels exactly from , since multiplying both masses by the same positive number leaves their ratio unchanged. Identify which errors are shared before assigning independent terms. If the desired result is final mass fraction , the stock mass fraction and its measurement uncertainty also belong in the measurement model.
For reference-assisted spectral offset separation, the corrected estimate depends on the measured sample change , reference change and sensitivity ratio . Its measurement uncertainty must include their joint variation, including uncertainty in a fitted sensitivity ratio. Subtracting a reference can reduce shared noise while adding independent noise.
The gradient of the measurement model with respect to is . If is their covariance matrix, first-order uncertainty propagation gives , orThe signs follow from the sensitivity coefficients, rather than a rule that all correlation reduces uncertainty. This is an application of the correlated-input propagation law in the primary GUM.[ref]
Evaluation of measurement data — Guide to the expression of uncertainty in measurement by JCGM. Section 5.2, equations (13)–(15), correlated input quantities
For an original independent-input example, take , , , , and . Then andTreating as exact would give only . A large reference response amplifies uncertainty in its conversion ratio, even when that response is measured precisely.
For mutually independent random inputs, the exact product variance also includes :where on the right denote the input means. It follows by expanding . This term is omitted by linearization. In the example it changes the standard uncertainty to about .
Experimentally, use paired sample/reference spectra to estimate shared covariance, and independent sensitivity measurements to test the proposed value of . If the same spectra estimate both and the correction, independence cannot simply be assumed. Drift between acquisitions or a nonlinear reference response is a measurement model problem as well as a precision problem.
Underdamped Langevin dynamics Created 2026-09-28 Updated 2026-11-05
Underdamped Langevin dynamics retains both position and momentum . For mass , temperature , friction rate and smooth potential energy , one common normalization isHere is vector Brownian motion. The noise amplitude is tied to friction by the fluctuation-dissipation relation for a Langevin particle. If the density is normalizable and the dynamics is nonexplosive, its invariant distribution is the Boltzmann distributionThe Hamiltonian flow preserves the total energy and phase-space volume; the momentum Ornstein-Uhlenbeck process preserves a centered normal distribution with coordinate variance . These two facts verify the invariant density. In dimensionless units with and momentum variance , it is , with the corresponding choice .
Retaining inertia resolves the short-time ballistic-to-diffusive Brownian displacement. The overdamped Langevin dynamics approximation removes momentum relaxation when only much longer times are relevant.
Unequal-noise allocation across calibration blocks distributes repetitions using block-specific variability while keeping the intended estimator fixed. Suppose independent blocks supply means and the target estimate is with specified weights . Let one observation in block have variance and equicorrelation within that block. ThenThe first sum is a block-specific floor; only the second decreases with repetition count. Changing the estimator weights merely to improve this calculation could change the estimand, so they remain fixed here.
For continuous positive allocations with positive costs , budget and , a Lagrange-multiplier calculation givesFor equal observation costs, repetitions are proportional to , not to the variance itself. Minimum counts and integer rounding must subsequently respect the actual acquisition constraints.
For an original two-block example, take , equal costs, , independent readings within each block, and standard deviations and in the output unit. The optimal counts are and , giving variance instead of from fifty each. If both within-block correlations are instead , the same proportions give variance , while equal counts give . The common-error floor now dominates and cannot be eliminated by taking more observations inside those same blocks.
This extends optimal allocation of detector repetitions across calibration blocks, whose balanced solution used homogeneous noise. For binary detector indicators, the per-reading variance is ; differing probabilities require a clearly specified weighted target. The planning variances and correlations must come from an appropriate measurement calibration model, not merely from counting files as independent blocks. NIST's uncertainty propagation retains shared-input covariance for exactly this reason.[ref]
Uniform distribution with an atom at zero Created 2026-10-18 Updated 2026-10-20
For and , this mixed probability distribution has probability mass at zero and ordinary probability density function on . That ordinary density integrates to , not to one; conditioned on , the law is the uniform distribution on . Its cumulative distribution function is zero for , for , and one for .
For each positive integer , integration gives . Thus the expected value is and the variance is . The nonzero atom contributes to normalization even though it contributes zero to positive moments. At , conditioning on positive delay is undefined because the conditioning event has probability zero.
An unbiased estimator is uniformly minimum-variance unbiased if its variance is no larger than that of every other unbiased estimator of the same parameter, for every parameter value. The Lehmann–Scheffé theorem gives such an estimator whenever an unbiased function of a complete sufficient statistic exists. Attaining the Cramér-Rao bound is sufficient for this property under its hypotheses, but is not necessary: for with , is unbiased and has variance , while the information bound is . Completeness follows from uniqueness of the Laplace transform of , so the estimator is still uniformly optimal among unbiased estimators.
A UMVCUE has minimum conditional variance among all estimators unbiased in the selected experiment for every parameter. For independent Gaussian stage estimates with information , continuation , total information and pooled estimate , Rao-Blackwellization of the fresh estimate gives , where . The conditional family has complete sufficient statistic , so the Lehmann–Scheffé theorem establishes the optimum. A conditional-likelihood bias correction does not automatically have this unbiasedness property.
Uniform order statistic Created 2026-09-28 Updated 2026-10-14
For independent and identically distributed random variables with the continuous uniform distribution on , the th order statistic has Beta distributionIts probability density function, mean, and variance areThe probability density function is zero outside ; values assigned to its endpoints do not change any probability. Peter Kempthorne's MIT lecture notes derive the uniform order statistic law and these moments. [ref]
https://ocw.mit.edu/courses/18-443-statistics-for-applications-spring-2015/06daf6d78e7eb0465d6bb53e8f96b2ff_MIT18_443S15_LEC11.pdf. Peter Kempthorne, MIT 18.443, Lecture 11, slides 12–14
.For , the minimum, sample median, and sample maximum correspond to . Their probability density functions are , , and , respectively. The sample median therefore has Beta distribution , mean , and variance . Its cumulative distribution function isFor instance, . The mean of the minimum is , and the mean of the sample maximum is . The fixed observed sample median in the order statistic example is one possible outcome; is the average over repeated statistical samples from this uniform distribution.
Each curve has area one. The extreme order statistics favor the ends, while the sample median favors the center. The endpoint density height is legitimate: the probability density function gives area per unit length, rather than point probability. In particular, a continuous uniform distribution has zero probability of producing an endpoint.
For a short proof, the binomial distribution counting identity under order statistic gives . Taking its derivative cancels consecutive terms, leaving the displayed probability density function. Recognize its normalizing factorial coefficient as that of the Beta distribution, then substitute , into the Beta distribution mean and variance formulas. Although the original observations are independent, their order statistics generally are dependent; see the positive covariance of two uniform order statistics.
Universal kriging Created 2026-10-05 Updated 2026-10-18
Universal kriging allows an unknown regression drift:The design matrix has full column rank, its entries and are known, and the regression coefficients are unknown. For example, rows and target row allow a linear spatial trend. Requiring to have zero bias for every gives . With known positive-definite matrix observation covariance matrix , the minimizing weights solvewhere is a vector of Lagrange multipliers. Equivalently, the generalized least squares estimate and prediction areThe first term is the estimated trend alone; the second predicts the correlated regression residuals. With , the error variance isThe last, nonnegative term accounts for estimating the specified drift coefficients. Merely inserting a fitted drift into simple kriging and keeping its known-mean error variance would omit this term. For a Gaussian random field, the same expressions arise by integrating over drift coefficients in a diffuse prior distribution limit; this is an additional inferential interpretation, not an assumption required by the linear optimization.[ref]
Gaussian Processes for Machine Learning by Carl Edward Rasmussen and Christopher K. I. Williams. §2.7, pp. 27–28, equations (2.38)–(2.42)
For a normal distribution with unrestricted and unknown , an integrable data-only unbiased estimator of the mean-vector prediction risk of a fixed rank- orthogonal projection matrix exists exactly when . To prove necessity, randomize the mean by independent Gaussian variance : conditioning adds to the squared bias term, whereas the marginal variance identity would add . When , the residual sum of squares itself is unbiased.
Unnormalized time autocorrelation Created 2026-10-06 Updated 2026-11-05
The raw same-observable time correlation . For a Stationary process with finite second moments, it equals . Its connected part is the autocovariance; unlike the normalized statistical autocorrelation, no division by the variance is imposed. The distinction matters for fluctuations around a nonzero equilibrium point.

