Rosenthal inequality Created 2026-09-24 Updated 2026-11-05
For independent centered variables and , the th moment of their sum is bounded, up to a constant depending only on , by the sum of their th moments plus the power of their total variance.
Rotational diffusion Created 2026-10-14 Updated 2026-10-18
Rotational diffusion is the random change of a particle's rigid body orientation through Brownian motion. A suspended ellipsoid can turn while its centre hardly moves; translational diffusion instead spreads its position. Random molecular collisions supply fluctuating torques as well as forces. The rotational diffusion coefficient measures how quickly angular uncertainty grows, rather than a steady spin speed. Its units are , often written to distinguish angular spreading from translational spreading, whose diffusion coefficient has units . The radian is dimensionless, but writing angular variance in remains useful.
First identify what the image can reveal. A marked end of an elongated particle defines an arrow: turning it through reverses that arrow. An unmarked ellipsoid with identical ends reveals only its long axis, a director: angles and describe the same visible state. An unmarked perfect sphere has no visible rigid body orientation at all. Tracking its centre cannot measure its rotational diffusion; a surface mark or another identifiable internal direction is needed.
This distinction makes rotational diffusion useful: it measures how long a particle remembers its pointing direction. For an elongated particle, turning also changes which laboratory direction is easiest for translational diffusion. Randomization of rigid body orientation can therefore erase directional preferences in the observed displacement, even though the particle's shape has not changed.
Rubin's rules Created 2026-10-06 Updated 2026-11-05
Under proper Multiple imputation with compatible imputation and analysis models, for completed-data estimates with variances , pool using , , , and . The total variance includes within-imputation uncertainty, between-imputation uncertainty and finite- simulation uncertainty.
Sample mean Created 2026-09-29 Updated 2026-11-05
For observations with , the sample mean is . It is a statistic summarizing location: values give . For an IID sample with mean and finite variance , linearity of expectation gives , while variance additivity for independent random variables gives . Dependence changes the variance through cross-covariances. A sample mean is sensitive to extreme observations, and a population such as the Cauchy distribution need not have the finite mean needed for ordinary averaging guarantees.
For independent and identically distributed Bernoulli random variables with success probability , the sum has a binomial distribution. Every length- pattern with successes has probability , and the binomial coefficient counts its possible success positions. Therefore
The usual endpoint convention includes and as deterministic cases. For and , the five probability masses are divided by . The sample mean takes values , with variance and standard deviation . At , its variance is and its standard deviation is .
Figure 1.
Exact sample-mean distributions distinguish fresh IID draws from a copied common coin
. Original exact binomial distribution probabilities for sample sizes . Blue marks give the mass at each possible mean under independent fair draws. Orange marks give the masses for the same fair draw copied times: at zero and at one for every . These are discrete probability masses, not density values or measured histograms. Details.
For the normal sample sum with known variance, assume independent and identically distributed with fixed known variance . The joint probability density factors as
The Fisher-Neyman factorization theorem proves that the sum, and hence the sample mean, is a sufficient statistic for . Its remaining deviations have a joint Gaussian law independent of and are independent of . If both and are unknown, the sample mean alone loses information about variance; the pair is sufficient for the full statistical parameter.
A discrete counterexample shows why a sample mean is not automatically sufficient for arbitrary statistical models. Let be independent and identically distributed on with
Each observation has mean one. The arrangements all give sample mean one, yet
This conditional probability is at and at . The conditional distribution therefore still depends on the unknown statistical parameter: this sample mean is not a sufficient statistic. The useful reduction depends on the family of full-data laws, even when its mean stays fixed.
Sample mean under simple random sampling Created 2026-10-14 Updated 2026-11-05
Fix real population values . Define the finite population mean and two normalizations of the finite population variance by
The values are fixed; only the selected subset is random. For its sample mean , the exact design expectation and variance are
The sample mean is an unbiased estimator of the finite population mean. At , every unit is observed and exactly, so the variance is zero. The case is deterministic and is handled directly, since is undefined then.[ref]
The finite population correction depends on the baseline. With the normalization it is multiplying . With the normalization it is multiplying , the exact variance for independent uniform sampling with replacement. The standard error uses the square root of the relevant variance factor. Mixing the two normalizations gives a wrong answer even at , when replacement cannot matter. For the six-value example, , , and , reproducing without replacement and with replacement.
A short proof exposes the statistical dependence. Let be the value at position in uniform sequential sampling without replacement, so each has expected value and variance . Write . Every ordered pair of distinct labels is equally likely, and , hence for ,
Linearity of expectation proves unbiasedness without requiring independence. The variance of a sum includes all ordered covariance terms, giving
For the original population, each pair of distinct draw positions has covariance . Drawing a high value leaves a lower mean among the remaining values, balancing subsequent draws. Equal marginal probability distributions therefore coexist with negative covariance.
When , the sample variance is an unbiased estimator of . To check this independently, use : its expected value is . Thus unbiasedly estimates the variance of the sample mean; its square root estimates the standard error, but need not be unbiased.[ref]
Figure 1.
Exact mean variance and expected distinct-unit coverage as sample size grows in the six-value population
. Details.
The diagram compares variance with the expected fraction of distinct population labels observed. Without replacement this fraction is exactly ; with independent uniform sampling with replacement, a label is missed with probability , so linearity of expectation gives expected fraction . At six draws, replacement covers only about of labels on average and retains sample mean variance ; sampling all six distinct labels covers the population completely and removes selection uncertainty. This population coverage is distinct from confidence interval coverage probability.
The correction matters when the sampling fraction is appreciable. It calibrates uncertainty about this fixed population under the specified selection mechanism. It does not correct unequal selection weights, missing observations or measurement errors; observing every label also does not eliminate uncertainty about a larger unobserved population. Moreover, an exact variance alone does not guarantee the coverage probability of a proposed confidence interval: a small discrete sampling distribution can make a normal approximation inaccurate.
Sample proportion Created 2026-10-07 Updated 2026-11-05
For independent Bernoulli observations with success probability , the sample proportion is the success count divided by the sample size. It is an unbiased estimator with variance . It supplies the arm-specific risks in a risk ratio and the mean difference in a two-proportion comparison.
Sample size for comparing two proportions Created 2026-10-07 Updated 2026-11-05
For equal independent binomial samples, the displayed normal-approximation size per group targets a two-sided significance level and power at specified alternative rates, with . Its first variance term sets the null critical value and its second the alternative distribution. A numerical sample size is not determined by an effect ratio alone; power, sidedness, allocation and baseline rates must also be specified. Rare-event discreteness and clustered designs can materially change the achieved power.
Sample standard deviation 2026-10-25
The sample standard deviation is the nonnegative square root of the usual sample variance for . The displayed convention uses divisor . Although the sample variance is an unbiased estimator of the population variance under independent identically distributed sampling with finite variance, taking its square root does not in general give an unbiased estimator of the population standard deviation.
A samplewise variance upper bound is a nonnegative statistic satisfying , where is the fixed sampling variance at parameter . It controls each realized value outside a probability-zero exceptional set. This implies the expected-value condition for a conservative variance estimator when the expectation exists; the converse fails, as the parent article's counterexample shows. The primary sampling-design definition only asserts the weaker expectation condition.[ref]
Suppose is an unbiased estimator with finite variance, and let . Then Chebyshev's inequality and the samplewise comparison give
No independence between and the centre is needed. If , unbiasedness makes the centre equal to almost surely, so coverage still holds. This proves confidence bounds from a samplewise variance bound; it does not justify replacing by a normal distribution critical value.
For an original bounded-data example, take independent and identically distributed measurements known to lie in , and estimate their population mean by the sample mean. With , the inequality gives
The deterministic choice is a samplewise variance upper bound for every such population. Thus covers with probability at least . The radius shows the price of distribution-free reasoning; the estimate may be much wider than a model-specific confidence interval.
Sampling an integer-shape gamma distribution Created 2026-09-24 Updated 2026-11-05
Let be a positive integer, , and independent uniform random variables on . Then
has a gamma distribution of shape and rate , also called an Erlang distribution. The unit interval is essential: an unspecified uniform distribution on another interval would not give this algorithm.
For , . Each summand therefore has an exponential distribution with rate , and additivity of independent gamma distributions with a common rate proves the result. For , the output has expected value and variance . The formula directly covers integer shapes; replacing by a noninteger number of draws does not define a sampler for a general gamma shape.
Sampling design 2026-11-05
A sampling design specifies a probability distribution over the possible samples from a defined population:
For sampling distinct labelled units, is a subset of the population. The design includes the selection mechanism, not merely the final sample size. A procedure with repeated draws can instead require a distribution over ordered sequences; the resulting inclusion probabilities concern whether a label appears at least once.
The design determines inclusion probabilities and joint inclusion probabilities through
These connect the physical sampling procedure to the Horvitz-Thompson estimator and its sampling variance.[ref]
An original comparison uses four fixed values and selects two units. Under simple random sampling, every one of the six pairs has probability . Each , so the Horvitz-Thompson estimator is twice the selected sum. It takes values with probabilities . Its expected value is the true finite population total , and its variance is
A second design chooses only or , each with probability . It has the same sample size and the same individual inclusion probabilities, but the estimator is now or , giving variance . Pair selection matters even when individual representation agrees.
A third design chooses uniformly among the four pairs containing one label from each of those two groups. Its total estimate is always for this particular population. This illustrates the potential gain from stratified sampling when groups correspond to relevant variation, not a claim that the third design dominates for every possible population. A reproducible design records the eligible labels and actual random selection rule before interpreting the observed sample.
Sampling fraction 2026-10-14
The sampling fraction is the sample size divided by the finite population size. In simple random sampling, it determines how much the finite population correction reduces the variance relative to .
Each Cartesian Brownian increment has variance . Summing their normalized squares gives a chi-squared distribution with degrees of freedom. Hence is unbiased and has relative standard deviation . Its inverse is not unbiased: for , . Thus the direct inverse-diffusivity radius estimate has appreciable finite-sample bias when only a few trajectories are observed. Measurement noise or drift adds separate systematic error.
Sampling variability 2026-11-05
Sampling variability is the variation of a statistic over random samples generated by a specified sampling design or model. It concerns the statistic before the sample is realized, even when the observed dataset is later treated as fixed.
For independent identically distributed observations with variance , the sample mean has variance and standard error . Thus increasing from 100 to 400 halves that standard error under the same model. A sampling distribution describes this variation; the bias of an estimator describes a displacement of the estimator's expectation from its target. Correlated observations or an informative selection mechanism can invalidate the simple independent-sample formula.
For the state-space model , , assume independent Gaussian process noise and measurement noise of variances , independent of the prior state and previous observations. A normal distribution for the filtered state, with mean and variance , predicts mean and variance . Here denotes predicted state variance, rather than the observation-noise covariance notation used in the vector equations above. Gaussian conditional expectation then gives posterior mean and posterior variance , where the Kalman gain is .
Mere weak white noise assumptions do not imply normal posterior distributions: independent process noise with a Rademacher distribution makes the next state a mixture of two shifted normal distributions. The Kalman filter can still describe a best affine state estimate under appropriate second-order assumptions, but that is distinct from an exact Gaussian conditional law.
Scalar rescaling invariance of a tie-line weight matrix means that multiplying every fit weight by the same positive number leaves the fitted phase fraction unchanged. Let endpoint compositions and overall composition be expressed in independent mole fraction coordinates. With , a fixed symmetric positive-definite matrix defines the tie-line phase-fraction fit by minimizing
For , . Multiplication by a positive constant preserves every comparison between objective values, so the sets of minimizers agree. The proof also applies when the same constraint is imposed in both fits.
For distinct endpoints, the unconstrained formula makes the cancellation visible:
The differential in sensitivity of a tie-line fit to its weight matrix agrees: for a perturbation , its numerator is by the fit's normal equation. Thus the scalar direction is insensitive even when the residual is nonzero.
In an original example, take , , and . The remaining composition coordinate is fixed by closure. These values give
Replacing by gives the same fraction, although the minimum objective increases from0.00072to0.00504. Changing only the second diagonal weight from4to7 instead changes the fitted fraction to0.640625. Relative weights determine the geometry; their common multiplier does not.
Point-estimate invariance does not by itself establish invariance of physical measurement uncertainty. Merely rewriting the same objective with leaves propagation through the same measured-input covariance matrix unchanged. If instead represents absolute precision and asserts a genuinely different covariance , that is a different noise model. With exact endpoints, the linear-model variance then decreases by a factor . The measurement model and covariance interpretation must be stated.[ref]
With query and key dimension , form the score matrix , apply the softmax function row by row, and multiply by the values . If query and key coordinates have independent unit variance, their dot product has variance ; division by keeps that score scale of order one. Optional additive masks exclude forbidden keys before the softmax function.
Scaled inverse chi-squared distribution Created 2026-09-28 Updated 2026-11-05
For parameters and , the Scaled inverse chi-squared distribution is the law of
The inverse change of variables gives its normalized density
It is an Inverse-gamma distribution with shape and scale . The label is a scale parameter, not the variance of .
Its mean is for , and its variance is for . Below those respective thresholds the positive moments diverge. For example, gives mean three and variance nine.
This is a conjugate prior for the unknown variance of a Normal distribution with known mean . With prior parameters and independent observations, the posterior has and . Multiplying prior and likelihood adds their powers and inverse-variance exponents, which proves the update. An unknown estimated mean requires a separate model and changes the update.