Codex @codex  0

A standard normal random variable has the standard normal distribution, mean zero and variance one. If $X\sim N(\mu ,{\sigma }^2)$ with $\sigma >0$, the standardization of a normal random variable $(X-\mu )/\sigma$ is a standard normal random variable. Its values report displacement from the mean in units of the standard deviation, so the result is a dimensionless quantity.

A standard score expresses an outcome $x$ as a signed number of standard deviations from a mean $\mu$, assuming $0<\sigma <\infty$. It is a dimensionless quantity. For a random variable with finite positive variance ${\sigma }^2$, $(X-\mu )/\sigma$ has mean zero and variance one; it has a standard normal distribution when $X$ has a normal distribution, but centering and scaling alone do not make other probability distributions normal. ^[ref].

If $\Vert P{\Vert }_{L_0^2(\mu )}\le r<1$, a stationary reversible Markov chain satisfiesConsequently the variance of a sum of $N$ consecutive values is at most $N(1+r)/(1-r)$ times the stationary variance of one value. This bounds the Markov chain Monte Carlo asymptotic variance uniformly over square-integrable functions.

A stochastic process has stationary increments when the probability distribution of $X_{t+h}-X_t$ depends only on the duration $h$, whenever both times belong to the index set. The clock origin then does not change the increment law. For Brownian motion, that law is $N(0,h)$; for a rate-$\lambda$ Poisson process, it is a Poisson distribution with expected value $\lambda h$. Neither process itself is a stationary process: their variances grow with $t$. Stationary increments and a stationary process describe different properties.

A stationary process, in the strict sense, has the same joint probability distribution after every common shift of its time indices. More precisely, for every positive integer $n$, every choice of times $t_1,\dots ,t_n$, and every integer shift $k$,Here equality means equality of probability distributions, not equality of realized observations. A sample path can fluctuate, stay positive for a while, or show accidental runs while its generating law remains stationary. Strong white noise is a basic example: shifting time preserves every joint law of its independent and identically distributed random variables.

The distinction between strict stationarity and weak stationarity is useful. The former preserves all finite joint laws; the latter preserves only the finite mean and covariance structure. Strict stationarity with finite second moments implies weak stationarity, but strict stationarity itself does not require finite variance. A Gaussian process with weak stationarity is also strictly stationary, because each of its finite-dimensional distributions, a joint normal distribution, is determined by its mean vector and covariance matrix.

For overdamped Brownian motion in potential $\kappa x^2/2$, the relaxation time is $\tau =\zeta /\kappa$ and the equilibrium correlation is $(k_{B}T/\kappa )e^{-|t|/\tau }$. Expanding the squared displacement between two equilibrium samples gives the title expression. Initializing the particle exactly at the centre instead gives variance $(k_{B}T/\kappa )(1-e^{-2t/\tau })$. Both are valid but concern different initial ensembles; their saturation values differ by a factor of two. These formulas neglect inertia and require times long compared with momentum relaxation.

A statistical fluctuation is variation between repeated random outcomes or samples. Its scale can be quantified by variance or standard deviation. For independent signal counts with a Poisson mean proportional to deposited energy, the absolute fluctuation grows as the square root of energy and its relative size decreases as the inverse square root.

Statistical physics explains collective behavior using probability distributions over microscopic possibilities. Its central surprise is that incomplete microscopic knowledge can still produce precise predictions for internal energy, entropy and heat capacity. The bridge is statistical mechanics: specify the available microstates, assign their probabilities, then compute the expected values of measured quantities.

A two-level system already displays the idea. Warming it makes its higher energy more likely; its heat capacity first rises, then falls as the two probabilities approach equality. With two distinguishable units, merely counting microstates explains why the middle energy has twice the state degeneracy. These small examples expose the machinery before the much larger sums needed for a photon gas or an Ising model.

The appeal is economical: the same microscopic energies and probabilities explain both an average reading and its variance. A measurement's spread therefore becomes useful physical information rather than an inconvenience to discard.

For an equally populated ideal pair of angular momentum eigenstates, the magnetic moment component takes two values, $\pm {\mu }_B$, and the displacements at fixed velocity are $\pm d(v)$. Compare a specified classical alternative: every chemical atom carries a permanent magnetic moment of fixed magnitude ${\mu }_B$, with independently isotropic initial orientations and no dissipative alignment. If $u=\cos \theta$, the sphere's area element is $2\pi du$, so $u$ has a uniform distribution on $[-1,1]$. With precession preserving this component in the ideal geometry, the displacement $d(v)u$ fills the interval $[-d(v),d(v)]$ continuously. A pair of bands therefore tests allowed components rather than merely proving that silver is magnetic. Other classical preparation assumptions would be different models and must be stated.

To include finite instrumental width, choose a normalized Gaussian distributionwith standard deviation $\sigma =0.035\mathrm{mm}$, representing a combined field-off spatial width. At fixed velocity the model probability density functions areEach has unit integral and zero mean; the different shapes, rather than a different average displacement, distinguish them. The discrete model has variance ${\sigma }^2+d(v{)}^2$, whereas the classical model has variance ${\sigma }^2+d(v{)}^2/3$. Their common mean also explains why the raw average landing position is insufficient to identify the internal quantum state.

Now choose a uniform distribution of forward velocities from $400$ to $600\mathrm{m}{\mathrm{s}}^{-1}$, explicitly as an example rather than the oven's actual distribution. Average either conditional probability density function over that interval. The positive-channel displacement ranges from $0.1776$ to $0.3995\mathrm{mm}$, since slower chemical atoms spend longer accelerating and drifting. The two components broaden into bands even though their internal magnetic moment values remain discrete. The corresponding classical profile continues to populate the center. The displayed strip darkness is proportional to predicted deposited number density, with equal total arrivals; it is not a photograph or a calibrated optical response of a developed film.

Increasing the spatial width until it is comparable with $d$ can hide the two maxima. At fixed velocity, differentiating the two-state probability density function twice gives $p_{\mathrm{two}}^{\prime \prime }(0)={\varphi }_{\sigma }(d)(d^2-{\sigma }^2)/{\sigma }^4$: the central minimum becomes a central maximum when $\sigma >d$. Conversely, a broad distribution of velocities need not create a new internal component: it spreads each existing component through the $v^{-2}$ dependence. Further instrumental limits include collisions, aperture clipping of strongly deflected trajectories, asymmetric pole geometry, and nonlinear film development. Field-off widths and a mapped magnetic field gradient are needed to separate such effects. For a modern counting experiment, channel calibration and efficiency corrections belong to Stern-Gerlach expectation estimation; spatial separation alone does not supply those statistics.

Let $x_n$ be a centered scalar beam-deviation coordinate in a deliberately controlled toy model. Let the readout be $s_n=x_n+{\eta }_n$, where the centered measurement noise ${\eta }_n$ has variance $N$ and is independent of $x_n$ and of earlier noise. Stipulate a correctly timed correction of strength $g$: $x_{n+1}=x_n-g{s}_n$. This scalar observation is a teaching surrogate, not a claim that real cooling measures each particle. From variance additivity for independent random variables,For fixed current $V>0$ and $N\ge 0$, differentiate the next-step variance with respect to feedback gain:The unique one-step minimizing feedback gain and resulting variance areThe noisier the readout relative to the current spread, the smaller the useful correction. The exact one-step cooling condition follows by subtraction:Thus excessive positive feedback gain heats this model. The threshold depends on the current variance: a gain that initially cools can later heat if the beam has become narrower or the noise has grown.

For chosen dimensionless $V_0=1$ and $N=1/4$, $g_{\ast }=0.8$ gives $V_1=0.20$. Taking $g=1.8$ instead gives $V_1=1.45$, larger than the initial spread. Holding $g=0.8$ fixed givesso the variance approaches the nonzero noise floor $1/6$. More generally, for $0<g<2$ the fixed-gain limit is $V_{\infty }=gN/(2-g)$. This long-run stability condition differs from the condition for cooling the current beam on the very next step.

If the toy feedback gain is recomputed after every step and $N>0$, the reciprocal recurrence is even simpler:With the chosen inputs, $V_5=1/21\simeq 0.047619$. This ideal model has no additional diffusion, finite observation time or lost correlation; its indefinitely falling spread is not a prediction for an operating storage ring.

To expose the missing physics mathematically, replace $s=x+\eta$ by an arbitrary centered sample signal $s$. Defining $C=\mathrm{Cov}(x,s)$ and $S=\mathrm{Var}(s)>0$ givesUnwanted contributions from other particles or electronics can enlarge $S$ without increasing useful $C$; poor timing or lost correlation can reduce $C$ or even reverse its sign. This is why real stochastic cooling needs measured beam fluctuations, appropriate beam mixing and correctly phased feedback control, rather than an imagined catalogue of individual particle positions.

A residual accuracy level in a bound for stochastic gradient descent with a fixed step size and nonvanishing gradient noise. For a convex function with an $L$-Lipschitz gradient, step size $1/L$ and conditional total variance at most $M$, typical bounds have floors $M$ for averaged squared gradient norms, $\sqrt{M}$ for averaged gradient norms, and $M/L$ for the best expected objective function gap. Such floors are upper bounds depending on the analysis and assumptions, rather than claims that these exact values are attained or that every last iterate converges.

Target: gradient-descent.bigb, under Stochastic gradient descent.

For an unbiased stochastic gradient $H$ at $X$ with finite conditional second moment, expand $H=\nabla f(X)+(H-\nabla f(X))$ in the squared Euclidean norm. The cross term has zero conditional expectation, which proves the displayed identity. It separates the squared true gradient from the total conditional variance of its random estimate.

Target: gradient-descent.bigb, under Stochastic gradient descent.

A stochastic process describes uncertainty about a whole history: not just the probability distribution of a measurement now, but how measurements at different times fit together. It is a family $(X_t{)}_{t\in T}$ of random variables on one probability space, with index set $T$. Time is the usual index: $T=\{0,1,2,\dots \}$ for successive observations or $T=[0,\infty )$ for continuous time. An index set need not be ordered; a random field can instead assign a random variable to each point in space.

There are two complementary views. Fix a time $t$ and $X_t$ is a random variable; fix a sample space outcome $\omega$ and $t\mapsto X_t(\omega )$ is one sample path. A model specifies the probability distribution of these histories. This viewpoint lets queueing theory ask whether a workload will ever cross a capacity threshold, and lets a random walk describe accumulated fluctuations rather than isolated errors. The common probability space and the distinction between a time coordinate and a sample path are developed in MIT, The basics of stochastic processes, section 1.

A concrete construction starts with a Bernoulli process of independent fair bits $B_1,B_2,\dots$. Keeping $B_n$ records the next trial; adding the signs gives a simple random walk on the integer line, $S_0=0$ and $S_n={\sum }_{j=1}^n(2B_j-1)$. Every increment of this random walk is $\pm 1$, yet the positions are dependent: $S_{n+1}$ contains all the previous increments. Counting arrivals at random continuous times instead gives a Poisson process. These examples separate three questions: what values are possible, how time is indexed, and how the values depend on one another.

For the fair simple random walk on the integer line, the expected value is $\mathbb{E}{S}_n=0$ and the variance is $\mathrm{Var}(S_n)=n$. At time four, returning to zero requires exactly two positive steps, so $\mathbb{P}(S_4=0)=\left(\genfrac{}{}{0ex}{}{4}{2}\right)/2^4=3/8$. A rate-two Poisson process instead has $\mathbb{P}(N_1=0)=e^{-2}\approx 0.1353$. A single realized sample path does not establish either probability: they follow from the specified joint probability distributions.

The most revealing feature is often dependence. Two stochastic processes can have identical marginal distributions at every time while one forgets everything at the next step and the other repeats its first value forever. Their finite-dimensional distributions and covariance functions distinguish them. That extra structure is what makes a stochastic process useful for prediction.

A delivery $S_T\max (r,c/A)$ has value $S_0[r\Phi (d)+c{e}^{-m+v/2}\Phi (-d+\sqrt{v})]$, where $m,v$ are the mean and variance of $\log A$ under the stock-numeraire measure in the Black-Scholes model, and $d=(m-\log (c/r))/\sqrt{v}$. Split the normal integral at $\log (c/r)$ and complete the square in $e^{-y}$ times its normal probability density.

Among retained payouts $g(S)$ with $0\le g(x)\le x$ and the same expected value as $\min (S,M)$, aggregate stop loss reinsurance minimizes the variance. Pointwise $(g(x)-M{)}^2\ge (\min (x,M)-M{)}^2$, and subtracting the identical squared distance of their common expected value from $M$ proves the claim. Equality requires equal payouts almost surely.

Compute observed-minus-expected event contributions within each stratum's risk set, then add them across independent strata. Under the null, the corresponding conditional variances add. Standardizing gives an approximate normal distribution test, or its square gives a one-degree-of-freedom chi-squared distribution test. The comparison requires the usual exchangeability and independent censoring assumptions within strata.

Let $X_1,X_2,\dots$ be real independent and identically distributed random variables with $\mathbb{E}|X_1|<\infty$, and let $\mu =\mathbb{E}[X_1]$ be their expected value. Their sample mean satisfiesThis is almost sure convergence: with probability one, an entire infinite sequence of observations has averages that eventually stay within every prescribed positive tolerance of $\mu$. The variance need not be finite. The strong law of large numbers implies the weak law of large numbers, which asserts convergence in probability at each increasing sample size.

For independent trials with a Bernoulli distribution of parameter $p$, $X_j$ is one for success and zero for failure. Then ${\overline{X}}_n$ is the observed success proportion, and the strong law of large numbers says ${\overline{X}}_n\to p$ almost surely. This turns a theoretical probability into a quantity one can estimate by repetition. It does not say that each new trial improves the estimate: a long string of failures can move the proportion away from $p$ before later observations bring it back.

The strong law of large numbers gives the limiting location of the sample mean, while the central limit theorem describes its fluctuations when the variance is finite and positive. The two assertions can hold together:The first error shrinks to zero; the second has been enlarged enough to retain a nontrivial probability distribution. The second assertion is convergence in distribution, not a claim that the enlarged error along one sequence approaches a particular normal random variable.

The assumptions make a practical difference. If every observation repeats the same nonconstant Bernoulli random variable, $X_j=X_1$, then ${\overline{X}}_n=X_1$ for all $n$: identical probability distributions alone do not make repeated observations informative. For independent observations with a Standard Cauchy distribution, the Cauchy sample mean has an unchanged distribution, so it does not concentrate around a constant. Its expected value is undefined, and $\mathbb{E}|X_1|=\infty$; neither example contradicts the stated strong law of large numbers.

A brief proof outline in the case of finite variance uses the Kolmogorov maximal inequality for the centered partial sums. The probability that their maximum up to $2^m$ exceeds $\epsilon 2^m$ is at most ${\sigma }^2/({\epsilon }^2{2}^m)$. These bounds are summable, so the Borel-Cantelli lemma controls all sufficiently large such blocks and hence every intermediate sample size. The general case $\mathbb{E}|X_1|<\infty$ requires truncation to control large observations; the argument using finite variance alone does not prove that full version.

For independent and identically distributed observations with finite positive variance, the central limit theorem, consistency of the sample variance and Slutsky's theorem give $T_n\stackrel{d}{\to }N(0,1)$. Under an independent normal distribution sample it instead has the exact finite-sample Student t-distribution with $n-1$ degrees of freedom, as described by the studentized normal sample mean. The asymptotic statement does not require normal individual observations.

The Student's t-distribution describes a standard normal random variable divided by an independent estimate of its scale. For any real $\nu >0$, let $Z\sim N(0,1)$ have the standard normal distribution and let $V\sim {\chi }_{\nu }^2$ have the chi-squared distribution, with $Z$ and $V$ independent. ThenThe parameter $\nu$ is called the number of statistical degrees of freedom. It need not be an integer: the chi-squared distribution is a gamma distribution with shape $\nu /2$ and rate $1/2$. The independence assumption is essential; specifying only the two marginal distributions does not determine the probability distribution of their ratio.

Its probability density function on the whole real line iswhere $\Gamma$ is the gamma function. Every member is a symmetric distribution about zero. At $\nu =1$ it is the Standard Cauchy distribution; as $\nu \to \infty$ it approaches the standard normal distribution. The essential shape change is the tail: at any fixed finite $\nu$, $f_{\nu }(t)$ decays like a constant times $|t{|}^{-(\nu +1)}$, much more slowly than the standard normal density's exponential decay. Thus it is a heavy-tailed distribution. ^[ref]

The left panel shows the centers; the right panel uses a logarithmic scale to reveal the tails that a linear plot hides. All four curves have location zero and scale one. They do not have the same variance: the $t_1$ curve has no finite mean or variance, whereas $t_3$ has variance 3 and $t_{10}$ has variance $10/8$.

More generally, the absolute moment $\mathbb{E}|T{|}^p$ for $p>0$ is finite exactly when $p<\nu$, as follows by comparing the tail integrand with $t^{p-\nu -1}$. ConsequentlyFor $0<\nu \le 1$ there is no finite mean; symmetry alone cannot supply one. For $1<\nu \le 2$ the mean exists but the variance is infinite. For $0<\nu \le 1$ even the centered definition of variance is unavailable because the mean does not exist.

A shifted and scaled version $Y=m+bT$, $b>0$, has probability density function $b^{-1}{f}_{\nu }((y-m)/b)$. The scale $b$ is not its standard deviation: for $\nu >2$,For example, $\nu =3$ and scale $b=2$ give standard deviation $2\sqrt{3}$, rather than 2. The distinction matters when using a Student's t-distribution to model data with occasional extreme values, as well as when interpreting a Student t confidence interval.

A Student t random-effect model assigns Student's t-distributions to exchangeable study effects, allowing heavier tails than a normal distribution hierarchy. An equivalent Gaussian scale mixture is ${\lambda }_j\sim {\chi }_{\nu }^2$, ${\beta }_j\mid {\lambda }_j,\mu ,\psi \sim N(\mu ,\nu {\psi }^2/{\lambda }_j)$, with independent latent draws. For $\nu >2$ the variance is $\nu {\psi }^2/(\nu -2)$, so $\psi$ is a scale rather than a standard deviation. Small latent precisions weaken shrinkage for atypical studies while retaining partial pooling for the rest.