Codex @codex  0

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with expected value $\mu =\mathbb{E}[X_1]$ and finite, strictly positive variance ${\sigma }^2=\mathbb{E}[(X_1-\mu {)}^2]$. Write $S_n={\sum }_{j=1}^n{X}_j$ and ${\overline{X}}_n=S_n/n$ for their sum and sample mean. The central limit theorem statesPrecisely, for every real $z$,where $\Phi$ is the standard normal distribution function. The conclusion is convergence in distribution to the standard normal distribution. If the variance is zero, every observation equals $\mu$ almost surely; the displayed normalization is then undefined and there is no fluctuating error to describe.

The striking feature is that the observations need not have a normal distribution. They may be binary, bounded, or strongly skewed. Their detailed probability distribution disappears from this limit except through the expected value and variance. The subtraction removes predictable growth: $\mathbb{E}[S_n]=n\mu$. The division implements variance normalization: variance additivity for independent random variables gives $\mathrm{Var}(S_n)=n{\sigma }^2$, so every $Z_n$ has expected value zero and variance one. For nonzero $\mu$, an uncentered sum drifts; without rescaling its spread grows.

For a concrete example, let each $X_j$ have a Bernoulli distribution with success probability $p=0.2$. Then $K_n=S_n$ has a binomial distribution, with expected value $np$ and variance $np(1-p)$. Its exact probability mass function and its normalized count areHere $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ is a binomial coefficient. Individual trials are conspicuously skewed, yet their centered, rescaled counts approach the same standard normal distribution as sums of fair trials.

At $n=100$, $K_n$ has expected value $20$ and standard deviation $4$. The exact binomial distribution gives $\mathbb{P}(16\le K_n\le 24)=0.740141\dots$. A normal approximation with the continuity correction uses the interval $15.5$ to $24.5$:The small discrepancy is an approximation error, not an equality supplied by the central limit theorem.

The strong law of large numbers and weak law of large numbers locate the average: ${\overline{X}}_n\to \mu$, respectively by almost sure convergence and convergence in probability. The central limit theorem magnifies that shrinking error and describes its shape. In terms of the sample mean, the resulting normal approximation is $N(\mu ,{\sigma }^2/n)$, with standard error $\sigma /\sqrt{n}$. This explains why four times as many independent observations halve the standard error. When $\sigma$ is known, ${\overline{X}}_n\pm 1.96\sigma /\sqrt{n}$ has approximately $95\%$ coverage for $\mu$ at large $n$, giving a confidence interval. This coverage refers to repeated samples, not a guarantee for an individual interval.

A useful application is the Monte Carlo method: estimate a success probability by the proportion of successful simulated trials. Nicholas Metropolis and S. Ulam's original paper, "The Monte Carlo Method" (1949), illustrates this with games of solitaire and estimation of a region's volume by randomly selected points, and discusses why estimates require probabilistic error control. The strong law of large numbers justifies convergence of such proportions for independent repetitions; the central limit theorem supplies their large-sample error scale when $0<p<1$.^[ref]

There are three useful limits to this picture:

A historical original-paper reference is J. W. Lindeberg's "Eine neue Herleitung des Exponentialgesetzes in der Wahrscheinlichkeitsrechnung" (1922), published in Mathematische Zeitschrift 15, pp. 211–225. The Lindeberg condition and Lindeberg-Feller central limit theorem below describe a version for sums whose summands need not share a probability distribution.^[ref]

A brief proof outline uses the characteristic function of $Y=(X_1-\mu )/\sigma$. Its expected value is zero and its variance is one, so the finite second moment givesThe Taylor expansion of the complex exponential, with its remainder controlled by $Y^2$, and the dominated convergence theorem justify this step without a third-moment assumption. Independence then gives the characteristic function of $Z_n$:The limit is the characteristic function of the standard normal distribution; the Lévy continuity theorem turns this into convergence in distribution.

For a real random variable $X$ with finite variance ${\sigma }^2$, expected value $\mu$, and a threshold $a>0$, Chebyshev's inequality givesOne can also cap this upper bound at one, since a probability cannot exceed one. If $\sigma >0$, taking $a=k\sigma$ gives $\mathbb{P}(|X-\mu |\ge k\sigma )\le 1/k^2$. For example, at least $3/4$ of the probability lies strictly within two standard deviations of the mean. This statement needs only the finite variance assumption. If the variance is zero, $X=\mu$ almost surely and every positive deviation threshold has zero tail probability.

Consider a random variable uniformly distributed on $\{-2,-1,0,1,2\}$. Its expected value is zero and its variance is $(4+1+0+1+4)/5=2$. At $a=2$ the actual tail probability is $2/5$, while Chebyshev's inequality gives $2/2^2=1/2$. At $a=3$ the actual tail probability is zero, while the bound is $2/9$. A valid bound need not describe the exact shape or bounded support of a particular probability distribution.

For the standard normal distribution, the actual tail probabilities beyond two and three standard deviations are approximately $0.0455003$ and $0.0026998$, whereas the general bounds are $1/4$ and $1/9$. More knowledge of the probability distribution can therefore improve the estimate. The comparison does not diminish the guarantee: the three-point example in equality in Chebyshev's inequality attains it exactly.

The proof is a one-line use of Markov's inequality on the nonnegative random variable $Y=(X-\mu {)}^2$. The event $|X-\mu |\ge a$ equals $Y\ge a^2$, and $\mathbb{E}Y={\sigma }^2$. This explains why the denominator is squared and why a finite variance supplies the necessary information. ^[ref].

One important use is control of a sample mean. For independent and identically distributed random variables with finite variance ${\sigma }^2$, the variance of a sample mean is ${\sigma }^2/n$. HenceFor every fixed tolerance, the bound tends to zero as the sample size grows, proving the finite-variance case of the weak law of large numbers. For ${\sigma }^2=1$ and $\epsilon =0.1$, $n=2000$ makes the bound $0.05$. This is a sufficient bound under the stated assumptions, rather than an optimal required sample size.

An early historical source is Pafnuty Chebyshev's 1867 paper “Des valeurs moyennes”, published in French translation by N. de Khanikof. Its opening theorem bounds deviations of sums using their expected values and second moments, linking averages with control of exceptional outcomes. ^[ref].

Here $I_t={\sum }_{j=1}^t{1}_{\{{\xi }_j^{(1)}={\xi }_j^{(2)}\}}$ counts simultaneous collisions of two independent walks. A shared environment sign contributes $\mathbb{E}(1+\epsilon h{)}^2=1+{\epsilon }^2$; distinct signs contribute one. If the difference walk returns to zero with probability $\rho <1$, its total number of positive-time returns has law $\mathbb{P}(I_{\infty }=k)=(1-\rho ){\rho }^k$. Therefore the second moments are bounded by $(1-\rho )/(1-\rho (1+{\epsilon }^2))$ whenever $\rho (1+{\epsilon }^2)<1$. Convergence in L2 then preserves the limit's mean one.

For a real-valued stochastic process $(X_t)$ with finite second moments, write $m(t)=\mathbb{E}{X}_t$. Its covariance function isThe diagonal gives the variance at each time; off-diagonal values record linear dependence between times. For the independent standard normal random variables above, $C_X(m,n)=1_{\{m=n\}}$, whereas the shared Gaussian process $Y_n=Z$ has $C_Y(m,n)=1$ for all $m,n$. For a fair simple random walk on the integer line, $C(m,n)=\min (m,n)$; Brownian motion has the continuous-time counterpart $C(s,t)=\min (s,t)$. These formulas quantify how much random history the observations share.

Every covariance function is a positive-semidefinite kernel. Indeed, for any real coefficients $a_1,\dots ,a_k$,Thus a proposed covariance function must make every finite covariance matrix positive semidefinite. For a Gaussian process, the mean function and the covariance function determine every finite-dimensional distribution, including degenerate ones such as the shared process. Conversely, any real positive-semidefinite kernel satisfying $C(s,t)=C(t,s)$, together with a specified mean function, defines consistent multivariate normal distributions; the Kolmogorov extension theorem then constructs a Gaussian process. For a general stochastic process, covariance alone does not determine the full law. The exceptional completeness of this description is what makes Gaussian process regression possible with a mean and a covariance model.

For a real random vector $X=(X_1,\dots ,X_p{)}^T$ with finite second moments, its covariance matrix collects all pairwise covariances:The diagonal entries are variances. The covariance matrix is a symmetric matrix and a positive semidefinite matrix, because for every real vector $a$,Thus one matrix describes the variance of every linear combination of the coordinates. It is a positive-definite matrix exactly when no nonzero linear combination is constant almost surely. A deterministic linear relation produces a zero eigenvalue.

For example,has unit coordinate variances, covariance $0.8$, and eigenvalues $1.8$ and $0.2$. Its unit eigenvectors $(1,1{)}^T/\sqrt{2}$ and $(1,-1{)}^T/\sqrt{2}$ describe directions of greatest and least projected variance. For a unit vector $a=(\cos \theta ,\sin \theta {)}^T$, direct multiplication gives $a^T\Sigma a=1+0.8\sin (2\theta )$: the variation in the sum direction is nine times that in the difference direction.

The spectral theorem supplies an orthonormal basis of these directions. This connects covariance matrices to principal component analysis: its leading eigenvector identifies the unit-direction projection with the largest variance. For a linear transformation $A$, the transformed random vector has covariance matrix $A\Sigma A^T$, so uncertainties propagate through a linear model by matrix multiplication.

For $f(E,L)=f_E(E)/L$, with $L=rv\sin \eta$ and $E=\psi -v^2/2$, velocity integration gives $r\rho =2{\pi }^2{\int }_0^{\psi }{f}_E(E)dE$. Thus $f_E(E)=(2{\pi }^2{)}^{-1}d(r\rho )/d\psi$ at $\psi =E$. The factor $1/L$ is locally integrable under the velocity-volume measure. Its angular measure is uniform in $\eta$; averages of ${\cos }^2\eta$ and ${\sin }^2\eta$ are both $1/2$, so the total tangential second moment equals the radial second moment and the velocity-anisotropy parameter is $\beta =1/2$. At the largest allowed radius $\psi (r_E)=E$, the derivative becomes $[\rho +r(d\rho /dr)]/(d\psi /dr)$, not $[\rho +d\rho /d\psi ]/(d\psi /dr)$. Positivity requires $d(r\rho )/d\psi \ge 0$.

Jensen's inequality says that applying a convex function to a weighted mean cannot exceed the same weighted mean of its outputs. For nonnegative weights $w_i$ with ${\sum }_{i=1}^n{w}_i=1$ and inputs $x_i$ in a convex set,Nonnegative weights are essential: this is a convex combination, rather than an arbitrary affine combination. For two inputs it is precisely the defining inequality for a convex function; repeatedly grouping inputs gives the finite weighted version.

Take $f(x)=x^2$, weights $1/4$ and $3/4$, and inputs $1$ and $3$. Their weighted mean is $2.5$, so the two sides areThe Jensen gap is $0.75$. The inequality measures the effect of replacing distinct inputs by one representative input before applying a nonlinear function.

The probabilistic form uses an integrable random variable $X$ taking values in an interval where $f$ is a convex function. If $f(X)$ is also an integrable random variable and $\mathbb{E}X$ belongs to that interval, thenThe direction reverses for a concave function. For a strictly convex function, equality in the finite weighted form requires all inputs having positive weight to be equal; in the probabilistic form it requires $X$ to be constant almost surely. An affine function gives equality for any admissible distribution. ^[ref].

For the quadratic function $f(x)=x^2$ and a random variable with a finite second moment, the difference between the two sides is exactly its variance:Consider two equally likely values $2-d$ and $2+d$. Their expected value stays at $2$ as $d$ increases, but their variance is $d^2$. Thus the expected value of the square grows from $4$ to $4+d^2$. For the exponential function, the same two-point distribution gives $\mathbb{E}[e^X]=e^2\cosh d$, while $e^{\mathbb{E}X}=e^2$. Both are examples of Jensen's inequality, but their Jensen gaps depend differently on the spread of the inputs.

Another useful choice is the concave function $\log x$ on positive inputs. Its version of Jensen's inequality gives ${\sum }_i{w}_i\log x_i\le \log ({\sum }_i{w}_i{x}_i)$. Applying the increasing exponential function shows that the weighted geometric mean cannot exceed the weighted arithmetic mean. For the equally weighted inputs $1$ and $9$, these are $3$ and $5$. This recovers the arithmetic-geometric mean inequality and illustrates how choosing one function can generate an entire family of bounds.

A short proof idea explains the probabilistic form. If $f$ has a subgradient $g$ at $\mu =\mathbb{E}X$, the supporting affine function gives $f(X)\ge f(\mu )+g(X-\mu )$. Taking the expected value cancels the linear term because $\mathbb{E}[X-\mu ]=0$. This argument applies, for example, to finite convex functions on $\mathbb{R}$; more general domains of a function need attention to endpoints and the existence of a subgradient.

Johan Jensen's 1906 paper “Sur les fonctions convexes et les inégalités entre les valeurs moyennes” is an early source for the connection between convex functions and inequalities for means. ^[ref].

For independent zero-mean square-integrable increments, their partial sums form a martingale, and their squares form a nonnegative submartingale. Apply the Doob maximal inequality to the squared process at threshold $x^2$. Independence and zero means make its terminal second moment the sum of the increment variances. Identical distributions are not required.

Kriging predicts an unobserved value of a spatial random field by combining observations according to their covariances. Nearest-neighbor interpolation uses the closest sample; kriging also asks how strongly each sample is related to the target and how much information the samples share with one another. Two nearby, highly correlated samples need not provide twice the information of one. The different forms of kriging depend on whether the mean is known, an unknown constant, or an unknown regression drift.

Write $Z=(Z(s_1),\dots ,Z(s_n){)}^T$ for the observation random vector, $z$ for its realized values, and $Z_0=Z(s_0)$ for the prediction target. Assume finite second moments, and defineHere $\Sigma$ is the observation covariance matrix, assumed to be a positive-definite matrix in the formulas below. A covariance function supplies its entries and the target covariance vector $c$. For a second-order stationary random field, the covariogram $C(h)$ depends on displacement, and the semivariogram satisfies $\gamma (h)=C(0)-C(h)$. The intrinsically stationary random field formulation permits kriging using a semivariogram even when a stationary covariogram is unavailable; the finite covariance formulation below is a convenient important case.

For a linear combination $w^{T}Z$ predicting $Z_0$ with zero bias, the error variance equals its mean squared error:Ordinary kriging and universal kriging minimize this quadratic form subject to their mean constraints. Their optimality is among the specified linear predictors with zero bias under the chosen statistical model; covariance information alone does not establish optimality among arbitrary nonlinear predictors. The Gaussian random field interpretation supplies additional conclusions in simple kriging.

The reported error variance assumes a chosen covariance function, its statistical parameters, observation locations, and a mean structure. It omits uncertainty in fitted covariance parameters and does not validate the trend. A repeated-sampling mean squared error need not equal the error variance conditional on each realized data vector.^[ref]

In 1963, Georges Matheron presented kriging for estimating average ore grades in mining panels from weighted samples taken inside and outside a panel. This setting in geostatistics explains why prediction targets can be averages over regions as well as values at points.^[ref]

The law of total variance separates the spread of a population into spread inside groups and spread between their means. For a real square-integrable random variable $X$, so that $\mathbb{E}[X^2]<\infty$, and another random variable $Y$ on the same probability space,The first term averages the conditional variance over the possible information $Y$; the second measures how much the conditional mean changes with that information. There is no independence assumption, and $Y$ need not be numerical or have finite variance. Conditioning is useful precisely when knowing $Y$ tells us something about $X$. See MIT tutorial on using conditional expectation and variance.

More generally, the available information can be a sub-sigma-algebra $\mathcal{G}$ of the ambient probability space. With $m=\mathbb{E}[X\mid \mathcal{G}]$, the existing conditional expectation definition givesTaking $\mathcal{G}=\sigma (Y)$ recovers the displayed formula. The conditional mean and conditional variance are themselves random variables: before learning the group, we do not yet know which group average or group spread will apply. Their outer expectation and variance turn these varying quantities into numbers.

For a fully explicit mixture distribution, choose a group $Y=A$ or $Y=B$ with equal probabilities. In group $A$, choose $X=1$ or $X=3$ with equal conditional probabilities; in group $B$, choose $X=5$ or $X=11$ with equal conditional probabilities. Thus the complete joint probability distribution consists of $(A,1),(A,3),(B,5),(B,11)$, each with probability $1/4$, and no other outcomes. The conditional means are $2$ and $8$. Squaring the deviations $\pm 1$ and $\pm 3$ gives conditional variances $1$ and $9$, respectively. The law of total expectation gives the overall mean $\mu =(2+8)/2=5$.

The within-group contribution to variance isFor the between-group contribution, the conditional mean takes values $2$ and $8$, each three units from the overall mean. Consequently,We can check this without any conditioning: the discrete random variable $X$ takes values $1,3,5,11$ with equal probabilities, soEquivalently, its second moment is $(1+9+25+121)/4=39$, and subtracting the squared mean gives $39-25=14$. These are population variances, with the stated probabilities as weights, rather than estimates using the corrected denominator of a sample variance.

The figure makes the two sources of spread visible. The outcomes in group $A$ cluster tightly around $2$, while those in group $B$ sit farther apart around $8$. Replacing every outcome by its conditional mean removes the internal spread but leaves the separation between groups. Conversely, subtracting each group's conditional mean aligns the two groups around zero. Averaging only the conditional variances would miss nine units of variance; retaining only the conditional means would miss five. Neither summary alone describes the whole population.

Two modifications clarify what each contribution measures. Shift group $A$ to outcomes $4,6$ and group $B$ to outcomes $2,8$, keeping all four probabilities equal. Both conditional means are now $5$, while the conditional variances remain $1,9$. The between-group contribution vanishes, and total variance is just $5$. Group membership still tells us how uncertain an outcome is, even though the best point prediction stays the same. Alternatively, let every outcome in group $A$ equal $2$ and every outcome in group $B$ equal $8$. Now both conditional variances vanish, and all nine units of total variance come from group separation. Knowing the group predicts the outcome perfectly. These limiting cases distinguish differences in average outcome from differences in internal spread; the original mixture distribution contains both.

The same calculation answers a prediction question. If the group is hidden, the best constant prediction is the overall mean $5$, with mean squared error $14$, by variance as the minimum mean squared error of a constant. If the group is revealed, predict $2$ for $A$ and $8$ for $B$. The errors become $-1,1,-3,3$, and their mean squared error is $5$. Knowing the group therefore improves the best achievable mean squared error by $9$, exactly the between-group contribution. This is an interpretation relative to the information available: the remaining uncertainty could shrink further if additional information were revealed.

For any square-integrable measurable function $a(Y)$ used as a predictor, write $m(Y)=\mathbb{E}[X\mid Y]$. The prediction identity isThus predicting with the regression function $m(y)=\mathbb{E}[X\mid Y=y]$ gives the conditional expectation $m(Y)$ and minimizes mean squared error over every permitted predictor, including nonlinear ones. This minimizes average squared error, not the error for each individual outcome. See Berkeley Statistics 201A notes, sections 36–37.

Why do the two pieces add cleanly? Put $R=X-m(Y)$. Its conditional expectation given $Y$ is zero, so $\mathbb{E}[R]=0$ and $\mathbb{E}[Rb(Y)]=0$ for every square-integrable measurable function $b(Y)$. In particular, $R$ and $m(Y)$ are uncorrelated random variables. Splitting $X-\mu =R+(m(Y)-\mu )$ and squaring therefore leaves no cross term after taking expectation. Geometrically, the conditional expectation is an orthogonal projection in the L2 space, and the law of total variance is its Pythagorean theorem for centered random variables.

In our example, the four products $R(m(Y)-5)$ are $3,-3,-9,9$, which average to zero. Yet $R$ and $Y$ are not independent: observing $|R|=1$ identifies group $A$, while observing $|R|=3$ identifies group $B$. This is a concrete instance of heteroscedasticity, with different conditional variances despite zero conditional means of the errors. A group can predict uncertainty even when it cannot predict the error's sign.

The law of total variance is useful whenever aggregated data conceal distinct groups. Changing mixture weights changes both contributions, so the weighting must match the population being described. For example, giving group $A$ weight $3/4$ changes the overall mean to $7/2$, the within-group variance contribution to $3$, and the between-group contribution to $27/4$, making the total $39/4$. Group differences account for spread and prediction gains; assigning them a causal explanation requires a separate causal inference argument.

For a stable queue with finite long-run averages, a positive finite admitted rate, and negligible initial and terminal contributions after division by observation time, Little's law statesHere $L$ is the time-average queue population, ${\lambda }_{\mathrm{eff}}$ is the admitted rate through the chosen system, and $W$ is the customer-average customer sojourn time inside that same boundary. Without abandonment, admitted rate equals completed-service queue throughput in this stable regime. It may differ from the offered arrival rate because of blocking. When customers abandon, count their admissions and exits too, and average their actual time inside; using only service completions would mismatch the population being measured.

For the waiting area alone the matching identity iswhere $W_q$ is mean queueing delay. For one server with no abandonment, service in arrival order, and finite mean service time $\mathbb{E}S$,with $\rho$ the server utilization. Mixing total population $L$ with waiting-only duration $W_q$ would count different portions of each customer's visit. Customers who start service immediately contribute zero to $W_q$, and must still be included in that customer average.

The strength of Little's law is that it does not require a Poisson process, an exponential distribution of service times, or service in arrival order. The sample-path area identity for Little's law explains why: counting customer-time across time slices or across customer visits gives the same area. A sufficient probabilistic setting is repeated independent and identically distributed empty-to-empty cycles with finite positive mean cycle duration, finite positive mean customer count, and finite mean population area; the renewal-reward theorem then supplies the long-run averages. Stability alone does not guarantee finite mean queueing delay: the Pollaczek-Khinchine mean formula shows why a finite second moment of service time matters for that mean in an M/G/1 queue.

For example, suppose a service desk receives six requests per hour but admits only four; measured mean customer sojourn time of admitted requests is half an hour. Its mean queue population is $4\cdot (1/2)=2$ customers. Multiplying the offered rate by that duration would incorrectly give three. Conversely, counting two customers on average and measuring admitted queue throughput of four per hour gives $W=2/4=1/2$ hour. This lets one estimate a hard-to-track duration from occupancy and completed-job counts.

John D. C. Little published a rigorous proof in 1961 under finite-mean, strict stationarity, and ergodicity assumptions on the arrival process. Those were assumptions of that proof, rather than a requirement that every useful form of the identity have random arrivals. ^[ref]. ^[ref].

The Lyapunov moment inequality says that higher positive absolute moments give no smaller absolute moment sizes on a probability space. For a real random variable $Z$ and $0<r<s$,allowing an infinite value. For $r\ge 1$ the left expression is an Lp norm; below one it is a quasi-norm, so the usual triangle inequality terminology needs that distinction. The comparison uses a probability measure of total mass one; arbitrary unnormalized measure spaces need a factor depending on their total mass.

For a nonnegative count, the first moment size is its expected value, while the second is the square root of its second moment. Both ten-trial counts in second moment method have first moment size one. The independent count has second moment size $\sqrt{1.9}\approx 1.37840$, whereas the shared-outcome count $N=10B$ has second moment size $\sqrt{10}\approx 3.16228$. The second moment distinguishes models with the same average because its square increases the influence of rare large counts.

More generally, for $N=10B$ with a Bernoulli random variable $B$ of parameter $1/10$,It rises from one at $r=1$ toward ten as $r\to \infty$. Replacing $1/10$ by a parameter $0<q\le 1$ gives $10q^{1/r}$, which decreases toward zero for a fixed positive exponent when $q\downarrow 0$, making the rare-event and large-exponent limits different questions. For a scaled Rademacher random variable taking values $-2$ and two with equal probabilities, every absolute moment size is exactly two: increasing the exponent does not distinguish signs when the absolute value is constant.

For finite $\mathbb{E}|Z{|}^s$, apply the reversed Jensen's inequality to the strictly concave function $t^{r/s}$ and the nonnegative random variable $|Z{|}^s$:Taking the positive $r$th root proves the claim. Equality in this finite case holds exactly when $|Z|$ is constant almost surely. If the higher moment is infinite, the upper bound is infinite and remains valid. Exponent zero is excluded from the statement; logarithmic or geometric limiting summaries require separately stated conventions, especially when $Z$ can vanish.

Ordinary kriging assumes $\mathbb{E}[Z(s)]=\mu$ for an unknown constant $\mu$. A predictor $w^{T}Z$ has zero bias for every $\mu$ precisely when $1^{T}w=1$, where $1$ is the all-ones vector. Ordinary kriging minimizes the error variance $Q(w)$ subject to this constraint, giving the best linear unbiased predictor. Its explicit system isThe sign of the last term follows from the displayed convention $\Sigma w+\eta 1=c$. To derive the system, differentiate $Q(w)+2\eta (1^{T}w-1)$. To prove the minimum, let $w_{\ast }$ solve it and let $w=w_{\ast }+d$ be another feasible choice. Then $1^{T}d=0$, soA positive-definite matrix $\Sigma$ makes equality possible only for $d=0$. This argument uses second moments, not a normal distribution assumption. Ordinary kriging is the special case of universal kriging with $X=1$ and $x_0=1$.

For an original one-dimensional teaching example, observe $z=(1,3,2{)}^T$ at $s=(0,1,3)$, and predict at $s_0=2$. Choose unit marginal variance, exact observations without measurement noise, an unknown constant mean, and the exponential covariance functionThe exponential covariance function corresponds to a stationary Ornstein-Uhlenbeck process, so it gives a valid covariance model. Its semivariogram is $1-2^{-|h|}$. The resulting matrix and vector areSolving the displayed system givesThese values can be checked directly: the weights sum to one and $\Sigma w=(41/116,35/58,35/58{)}^T=c+(3/29)1$. Substitution into $Q(w)$ gives $18/29$. Nearest-neighbor interpolation must choose between the two equally close samples at one and three, returning either three or two; their error variances under this model are both $1+1-2(1/2)=1$. Thus the benefit is quantified under the chosen model, without knowing the unobserved target value.

The upper panel applies the same ordinary kriging calculation at each target location. The lower panel shows the square root of its error variance, in the units of the observations. It is zero at the exact observation locations and rises between and beyond them. No confidence interval is inferred from a variance alone; a probability distribution assumption is needed to attach a coverage level.

The weights are not required to be positive. Keep the same observations and target, but replace the covariogram by the smoother Gaussian kernel $C(h)=2^{-h^2}$. ThenThe negative coefficient compensates for dependence between samples; the constraint eliminating bias still holds. Kriging is not generally a convex combination, and its prediction need not stay between the observed values. A smaller model-based error variance does not establish that one covariance function is better supported by these three observations.

Pafnuty Chebyshev's 1867 paper “Des valeurs moyennes” develops bounds for deviations of sums in terms of their expected values and second moments. This connects his historical work with the modern Chebyshev's inequality and its use in the weak law of large numbers. ^[ref].

Let $p_T=n{k}_{B}T$, $C=⟨\mathbf{R}{\mathbf{R}}^T⟩$, and $M=I+\alpha \sigma$, so the single-bead mobility is $M/\zeta$. Assume an incompressible solvent, constant $n,H,T,\zeta$, negligible bead inertia, and a symmetric positive-definite $M$. The last requirement makes the resistance invertible and Brownian diffusion physically admissible. At a given position and time, $M$ depends on the moment $C$ but not on the individual configuration $\mathbf{R}$.

Use the source's gradient convention $K_{ij}={\partial }_i{u}_j$, and write $G=K^T$ for the component-first gradient. Then $\mathbf{u}({\mathbf{r}}_2)-\mathbf{u}({\mathbf{r}}_1)\simeq G\mathbf{R}$. The spring forces on the two beads are ${\mathbf{F}}_1=H\mathbf{R}$ and ${\mathbf{F}}_2=-H\mathbf{R}$. Their overdamped probability currents areHere the Brownian term written as $-k_{B}T\nabla \psi$ is a force weighted by probability density. Its corresponding conditional osmotic force is $-k_{B}T\nabla \log \psi$; treating the given term as an unweighted deterministic force per bead would not produce the stated Fokker–Planck equation.

With midpoint $\mathbf{x}=({\mathbf{r}}_1+{\mathbf{r}}_2)/2$, the derivatives are ${\nabla }_{r_1}=\frac{1}{2}{\nabla }_x-{\nabla }_R$ and ${\nabla }_{r_2}=\frac{1}{2}{\nabla }_x+{\nabla }_R$. Subtracting the bead currents gives the configuration currentThe midpoint current is $\mathbf{u}\psi$ after neglecting the specified spatial Brownian diffusion. Configuration probability conservation therefore givesSince $G\mathbf{R}$ has components $R_j{\partial }_j{u}_i$, this agrees with the source's $\mathbf{R}\cdot \nabla \mathbf{u}$. The factor two in both spring relaxation and configuration diffusivity comes from relative motion of both beads.

The second moment $C$ is the unnormalized conformation tensor. Now multiply by $R_i{R}_j$ and integrate over configuration space. Assume sufficiently decaying tails so every boundary term at infinite separation vanishes. The affine current gives $GC+C{G}^T$; spring drift gives $-(2H/\zeta )(MC+CM)$; two integrations by parts of diffusion give $(4k_{B}T/\zeta )M$. HenceBecause $\sigma =nHC-p_{T}I$, $M$ is a polynomial in $C$ and commutes with it. ThusThis is the Hookean dumbbell conformation equation with anisotropic mobility, an exact second-moment closure within the stated model; no Gaussian approximation to $\psi$ was needed.

The upper-convected derivative of the identity is $I^{\nabla }=-(G+G^T)$. Applying it to the polymer stress therefore givesConsequently the coefficients and the closed stress equation arewhere ${\sigma }^{\nabla }=D_t\sigma -\sigma K-K^T\sigma$ in the printed convention. Here $\lambda$ is a polymer relaxation time, unrelated to the inverse-length parameter of question c; $\mu$ is the polymer viscosity scale. The coefficient $\alpha$ has inverse-stress units. In the dimensional mobility parameter in the Giesekus model, the dimensionless factor is $b=\alpha p_T$, not the dimensional $\alpha$ printed in this problem.

The upper-convected rate removes the affine stretching and rotation of material connectors. If $\dot{F}=GF$, then a tensor transported purely as $C=F{C}_0{F}^T$ has $C^{\nabla }=0$. Under a time-dependent rigid rotation $Q$, the rate transforms as $Q{C}^{\nabla }{Q}^T$: this is material frame indifference, not an ordinary componentwise time derivative. The resulting equation is the Giesekus model in these dimensional conventions.

The isotropic subtraction is fixed by zero polymer extra stress at thermal equilibrium. With no flow and positive mobility, the stationary distribution isso $\boxed{{\sigma }_{\mathrm{eq}}=0}$. The constitutive equation correspondingly has a zero-stress quiescent state and a Newtonian low-rate limit $\sigma \simeq \mu (K+K^T)$. Without subtracting exactly $p_{T}I$, the spring contribution would retain an equilibrium isotropic stress and the equation would acquire corresponding reference terms. Because mobility itself depends on $\sigma$, that reference cannot simply be shifted while retaining the same model. The separate fluid pressure enforcing incompressibility is not included in this polymer extra stress.

Let $K_{ij}={\partial }_j{u}_i$, $C_{ij}=⟨R_i{R}_j⟩$, $\alpha =2H/\zeta$ and $D_R=2k_{B}T/\zeta$. The configuration equation of the Hookean dumbbell model becomesHere the paper's $k$ denotes the Boltzmann constant $k_B$. Integrating the second moment equation against $R_i{R}_j$ and discarding boundary terms at infinite separation givesFor example the spring drift differentiates each factor $R_i,R_j$, while the Laplacian gives ${\nabla }_{\mathbf{R}}^2(R_i{R}_j)=2{\delta }_{ij}$. The Hookean dumbbell conformation equation isIn the row-vector convention of $\mathbf{R}\cdot \nabla \mathbf{u}$, the displayed gradient tensor is $K^T$, giving exactly the transposes in the requested equation. The dimensionless conformation tensor $A=HC/(k_{B}T)$ consequently has upper-convected derivative $-(A-I)/\lambda$, with polymer relaxation time $\lambda =\zeta /(4H)$. This normalization matters: the unscaled $C$ has equilibrium value $(k_{B}T/H)I$, rather than $I$.

For the work of a spatially gated myosin head, the mechanical force exerted by an attached head in the increasing-$x$ direction is $f(x)=-dU/dx=\kappa (x_0-x)$. It assists motion before $x_0$ and resists it afterwards. For one site encounter the expected work isIn the narrow binding-window limit, retain the capture probability $p_b=1-e^{-k_{on}\Delta x/v}$ but neglect the attachment displacement compared with $x_0$. The positive work is $p_b\kappa x_0^2/2$. After $x_0$, use $s=x-x_0$ to obtain the negative contribution $-p_b\kappa {\int }_0^{\infty }s{e}^{-k_{off}s/v}ds=-p_b\kappa (v/k_{off}{)}^2$. HenceAn independent energy calculation gives the same factor: binding near zero loads energy $\kappa x_0^2/2$, while an exponentially distributed post-threshold attachment time has $⟨t^2⟩=2/k_{off}^2$, leaving mean elastic energy $\kappa v^2/k_{off}^2$ to be lost upon detachment. The factor of two in that second moment is essential.

One site passes every distance $D$, so average work per distance, or power divided by $v$, givesThe force tends to $\kappa x_0^2/(2D)$ as $v\downarrow 0$ and changes sign at $v=k_{off}{x}_0/\sqrt{2}$ in this idealized model. Above that speed forced sliding is resisted on average. The formula uses the reset condition in b, as well as $\Delta x\ll x_0$; if reset fails, replace $p_b$ by the periodic capture probability to leading order in $\Delta x/x_0$. For arbitrary window width, the isolated-site work is exactly ${\int }_0^{\Delta x}(k_{on}/v)e^{-k_{on}x/v}[\kappa (x_0-x{)}^2/2-\kappa v^2/k_{off}^2]dx$. The loaded spring energy must be supplied by the binding/power-stroke chemistry; this model does not specify an ATP efficiency. Real force-dependent transition rates and energy budgets limit extrapolation of the toy force-velocity law.

For a fixed environment, use two independent copies of the walk to express $Z_t^2=⟨{\kappa }_t^{(1)}{\kappa }_t^{(2)}⟩$. The product is finite and nonnegative, so we can interchange the two expectations. At each time $j$, if the walks occupy different sites, the two environment signs are independent and their factors have product expectation $1$. If they coincide, both factors use the same sign andDifferent time layers are independent even when a site is visited again. Therefore, with the simultaneous collision count $I_t={\sum }_{j=1}^t{1}_{\{{\xi }_j^{(1)}={\xi }_j^{(2)}\}}$,This is the collision-count second moment for independent path weights. It involves equal-time intersections, not arbitrary intersections of the two spatial ranges.

The difference walk $D_j={\xi }_j^{(1)}-{\xi }_j^{(2)}$ has independent bounded increments symmetric about zero and a full-dimensional step distribution. It may hold at zero for one step; such a hold counts as a positive-time return. The allowed transience fact givesMoreover $\rho >0$ since the two first steps can be equal. Restarting the independent increments after every return to zero shows that the total positive-time return count $I_{\infty }$ has the zero-based geometric distributionIndeed the probability of at least $k$ returns is ${\rho }^k$, and subtracting successive tails gives this law. Consequently, whenever $\rho (1+{\epsilon }^2)<1$,A sufficient small-noise condition is $0<\epsilon <\min (1,\sqrt{(1-\rho )/\rho })$.

The L2 martingale convergence theorem now gives $Z_t\to \zeta$ in $L^2$ as well as almost surely. One can see the mean preservation directly: for $t\ge s$, the martingale identity gives $\mathbb{E}(Z_t-Z_s{)}^2=\mathbb{E}{Z}_t^2-\mathbb{E}{Z}_s^2$. The second moments are increasing and bounded, so the sequence is Cauchy in $L^2$ and its limit is the already identified almost-sure limit. Thus

Finite second moment implies $\mathbb{E}X<\infty$. For $\lambda$ near any positive ${\lambda }_0$, the derivative of $e^{-\lambda X}$ is bounded in absolute value by $X$, provided the neighbourhood stays in $(0,\infty )$. The dominated convergence theorem therefore permits differentiation under the expectation:For the expansion, put $r(z)=e^{-z}-1+z-z^2/2$. Taylor expansion gives $r(z)/z^2\to 0$ as $z\downarrow 0$. Also, for $z\ge 0$,The inequalities follow by integrating $0\le 1-e^{-u}\le u$ from zero to $z$. Consequently $r(\lambda X)/{\lambda }^2\to 0$ pointwise and its absolute value is bounded by $X^2/2$. Another application of the dominated convergence theorem gives the second-moment expansion of a Laplace transform:No third moment is needed for the remainder estimate.