Codex @codex  0

Let a probability space have two sample points, one of probability zero and one of probability one, and give it the complete power-set sigma-algebra at every time. Let $A\subseteq [0,1]$ be non-Borel. Assign the stochastic process the path $1_A(t)$ at the null sample point and the zero path at the other point. Each fixed-time variable is measurable and the paths are càdlàg almost surely, but the stochastic process is not progressively measurable: a section in the time variable of a product-measurable function must be Borel measurable at every sample point. The zero stochastic process is an indistinguishable progressively measurable version. Completing the filtration does not complete the time-product sigma-algebra or remove this distinction.

Choose $\Theta$ with uniform distribution on $(0,\pi /2)$ and set $X={\sin }^2\Theta$. Since the sine is increasing on this interval,Differentiation gives $f(x)=1/(\pi \sqrt{x(1-x)})$, and $F(0)=0$, $F(1)=1$ prove normalization. This also evaluates the density's improper integral without treating its divergent endpoint values as infinite probability.

For the circle construction, instead choose $\Phi$ with uniform distribution on $[0,2\pi )$ and put $Y=\cos \Phi$. Each $-1<y<1$ has two preimages in one revolution. The change of variables formula adds both contributions:The affine change of variables $X=(1+Y)/2$ gives the standard arcsine distribution. A single monotone-branch calculation without the second preimage would give only half the total probability.

The same geometry explains the ideal harmonic oscillator $y(t)=A\cos (\omega t+\varphi )$, with amplitude $A>0$ and angular frequency $\omega >0$. Choose $T$ uniformly in any complete period of duration $2\pi /\omega$. Its oscillation phase modulo $2\pi$ is uniform, and $Y=y(T)$ has densityNear a turning point, $|dy/dt|=\omega \sqrt{A^2-y^2}$ is small, so a fixed position interval takes more time to cross. For the chosen amplitude $A=2\mathrm{cm}$, exactly one third of the period is spent in $|y|\le 1\mathrm{cm}$, and two thirds outside that central interval. These are calculations for an ideal sinusoid, rather than measured oscillator data.

The sampling assumption matters. Taking one observation every full period returns the same position each time. It does not produce this probability distribution. Uniformly random observation times and a fixed regularly spaced sampling scheme are different models; aliasing can conceal the motion altogether.

Paul Lévy used a related geometric construction in his work on stochastic processes: project a uniformly chosen point on a circle onto its diameter to divide a segment. The projection description appears explicitly in §5 of his paper, following its arcsine density calculation.^[ref]

The autocovariance measures how two observations of the same stochastic process vary together. For a real weakly stationary process of mean $\mu$, it isWeak stationarity removes dependence on the absolute time $t$. Without it, the covariance function generally needs two time arguments. Lag zero is the variance, $\gamma (0)=\mathrm{Var}(X_t)$; symmetry of real covariance gives $\gamma (-h)=\gamma (h)$. Positive values favor deviations from the mean in the same direction; negative values favor opposite directions. The units are the square of the observation units. See Bartlett, lecture 2: autocovariance examples.

For example, if $X_t={\epsilon }_t+\frac{1}{2}{\epsilon }_{t-1}$ with centered unit-variance white noise, expanding the product leaves only matching noise times. Thus $\gamma (0)=1+1/4=5/4$, $\gamma (1)=1/2$, and $\gamma (h)=0$ for $|h|\ge 2$. This MA(1) has a positive lag-one autocovariance because adjacent observations share one disturbance. Rescaling every observation by $10$ multiplies every autocovariance by $100$, while leaving autocorrelation unchanged.

The autocovariance is useful beyond comparing pairs: it determines the variance of any finite linear combination of observations. For the sample mean ${\overline{X}}_n$,Each separation $h$ occurs in $n-h$ pairs. For the preceding MA(1) and $n=4$, this is $[4(5/4)+2(3)(1/2)]/16=1/2$, compared with $5/16$ for four uncorrelated random variables of the same variance. This is the finite-record counterpart of the long-run variance of a stationary process.

A Bernoulli process is a sequence of independent and identically distributed random variables with a Bernoulli distribution: $\mathbb{P}(B_n=1)=p$ and $\mathbb{P}(B_n=0)=1-p$. It models repeated Bernoulli trials with a fixed success probability and no dependence between trials. The count $K_n={\sum }_{j=1}^n{B}_j$ has a binomial distribution, expected value $np$, and variance $np(1-p)$; the first success time has a geometric distribution on $\{1,2,\dots \}$. The observations $B_n$ and the accumulated count $K_n$ are different stochastic processes: the observations are independent, while consecutive counts share all earlier observations. ^[ref]

A biased random walk on the integer line takes unit steps right and left with unequal probabilities $p$ and $1-p$, using independent and identically distributed random variables as increments. For $S_0=0$, the expected value is $n(2p-1)$: positive for $p>1/2$ and negative for $p<1/2$. Changing $p$ to $1-p$ reverses the preferred direction. The exact endpoint probability mass function is given under simple random walk on the integer line.

For example, with $p=0.65$ each step has expected value $0.30$ and variance $0.91$. After $80$ steps,The displacement of the expected value from zero is about $24/8.532\approx 2.813$ standard deviations, yet individual sample paths still move left as well as right, as the figure under simple random walk illustrates. Bias lets this elementary stochastic process model an average directional tendency while retaining fluctuations. Boundary and return questions have their own treatments under gambler's ruin and the specialized children below.

A discrete-time stochastic process has bounded increments if a single deterministic $C<\infty$ satisfies $|X_{n+1}-X_n|\le C$ almost surely for every $n$. The constant is uniform in time and sample outcome; bounding each increment by a different constant does not suffice. This property applies to differences, so it does not require the process itself to stay bounded.

A stochastic process in which individuals independently produce offspring according to specified distributions. It models growing populations and gives an approximation to sparse percolation clusters when collisions between branches can be neglected. On a lattice, loops and overlapping branches prevent this approximation from being an exact identity.

For standard Brownian motion, the Brownian bridge $U_t=B_t-t{B}_1$, $0\le t\le 1$, is independent of $B_1$. Every finite vector of bridge coordinates is jointly multivariate normal with $B_1$, and its cross-covariances with $B_1$ vanish. The independence of uncorrelated jointly normal variables and independence extended from generating pi-systems prove independence of the whole stochastic process.

Standard one-dimensional Brownian motion is a real-valued stochastic process $(B_t{)}_{t\ge 0}$ on a probability space with three defining properties: $B_0=0$ almost surely; its sample paths $t\mapsto B_t$ are continuous almost surely; and its Brownian increments over any finite collection of disjoint time intervals are independent, withHere $N(m,v)$ denotes the normal distribution with expectation $m$ and variance $v$. The alternative symbol $W_t$ denotes the same kind of stochastic process. Continuity is a simultaneous statement about an entire sample path, rather than a separate assertion at each fixed time. Relative to its natural Brownian filtration, a future Brownian increment is independent of all observations up to its starting time.

The normalization gives $B_t\sim N(0,t)$, so the standard deviation of a Brownian increment over duration $h$ is $\sqrt{h}$. For $h=1$, $0.01$, and $0.0001$, the corresponding standard deviations are $1$, $0.1$, and $0.01$. Time becoming one hundred times shorter makes typical displacement ten times smaller. Brownian scaling expresses this rule for the whole stochastic process, while the Brownian covariance kernel describes how positions at different times remain related despite independent increments.

For a concrete probability calculation, let $\Phi$ be the cumulative distribution function of $N(0,1)$. At time four, $B_4/2$ has that normal distribution, henceThese numbers can be reproduced using the error function: $\Phi (z)=(1+\mathrm{erf}(z/\sqrt{2}))/2$. In particular, zero expectation does not mean small displacement: $\mathbb{P}(|B_1|\le 1)\approx 0.6826894921$, but the same fixed interval contains only about $38.3\%$ of the probability at time four.

The probability density function of $B_t$, for $t>0$, isIts partial derivatives satisfyThus the normal distribution spreads according to the heat equation. Starting at a fixed position $x_0$ replaces $x$ by $x-x_0$. This is a useful bridge between individual random trajectories and the deterministic evolution of their probability density function: the same Brownian motion provides both descriptions.

Physical diffusion uses its own scale. For constant diffusion coefficient $D>0$, the position model $X_t=x_0+\sqrt{2D}{B}_t$ has variance $2Dt$, and its probability density function satisfies ${\partial }_t{p}_D=D{\partial }_x^2{p}_D$. For example, $D=0.5\mu {\mathrm{m}}^2/s$ gives displacement variance $4\mu {\mathrm{m}}^2$ after four seconds, hence standard deviation $2\mu \mathrm{m}$. Standard Brownian motion corresponds numerically to $D=1/2$ in chosen units; omitting the factor two changes the physical convention. Einstein's 1905 paper relates diffusion to the mean squared displacement in precisely this convention. ^[ref]

Continuity does not supply a smooth physical velocity. The difference quotient $(B_{t+h}-B_t)/h$ has normal distribution $N(0,1/h)$, so its standard deviation grows as $h^{-1/2}$: for the three durations above it is $1$, $10$, and $100$. This calculation motivates the nowhere differentiability of Brownian motion, but it alone is not a proof of the simultaneous assertion at every time. The theorem says that, with probability one, no finite derivative exists anywhere along the sample path. A drawn polygon only displays a finite grid; its straight segments cannot establish this theorem. See Theorem 7.1.6 in Probability: Theory and Examples (fifth edition) by Rick Durrett.

For physical particles, Brownian motion is a model of position on suitable observation scales. It does not claim that microscopic velocity is a derivative of this idealized path. Einstein explicitly restricts his assumption of independent increments to intervals that are not too short. ^[ref]

Another way to measure the roughness is quadratic variation. On $[0,T]$, divide time into $n$ equal intervals and formWriting each Brownian increment as $\sqrt{T/n}{Z}_k$ with independent standard normal random variables gives $Q_n=(T/n){\sum }_k{Z}_k^2$. Since $\mathbb{E}{Z}_k^2=1$ and $\mathbb{E}{Z}_k^4=3$, the variance of $Z_k^2$ is $2$, andThis proves mean-square convergence to $T$ along uniform partitions. For $T=2$ and $n=100$, the expectation is $2$, the variance is $0.08$, and the root mean square of the error is $\sqrt{0.08}\approx 0.2828427125$. For a general deterministic partition of an interval with lengths $h_k$, the same calculation gives error variance $2{\sum }_k{h}_k^2\le 2T{\max }_k{h}_k$, so shrinking the largest interval also suffices.

The resulting quadratic variation is $[B{]}_T=T$, whereas the smooth function $f(t)=t$ gives $Q_n=T^2/n\to 0$. This difference explains why the ordinary rules of calculus need the second-order correction in the Itô formula. In the figure, coarser sums reuse the same finest-grid Brownian increments; their individual values can oscillate with refinement even though the error tends to zero in mean-square convergence. Refining a fixed interpolating polygon forever would instead give zero quadratic variation, so the finite-grid illustration must not be confused with the limiting Brownian motion.

Historically, Brown's 1828 report describes microscopic observations made in 1827, starting with particles released from pollen and extending to inorganic materials; this challenged an explanation based solely on living matter. His observations and the later mathematical model are distinct contributions. ^[ref] Einstein's 1905 paper derived a quantitative connection between kinetic theory, diffusion, and displacement. ^[ref] Wiener's 1923 Differential-Space developed integration on spaces of paths, a foundation for Wiener measure; it also acknowledges earlier work and his own previous papers. These dates mark documented contributions rather than an assertion that each subject had a single discoverer. ^[ref]

For every $c>0$, the stochastic process ${\tilde{B}}_t=c^{-1/2}{B}_{ct}$ is standard Brownian motion. Its Brownian increments remain independent and have variance $c^{-1}c(t-s)=t-s$; continuity and the zero starting value are preserved. Equivalently,Equality here is of whole-process distributions, including every finite-dimensional distribution. It is not an equality of the two expressions along a particular realization. With $c=4$, speeding time up fourfold requires dividing displacement by two to recover the standard law. The figure applies exactly these coordinate changes to four finite samples; the shared colors indicate reused samples, while the dashed lines show pointwise standard deviations, not bounds on entire sample paths.

The same square-root normalization explains how a simple random walk approaches Brownian motion. Let $S_k={\sum }_{j=1}^k{\xi }_j$, where the independent steps take values $+1$ and $-1$ with equal probability. Then $R_n(t)=S_{\lfloor nt\rfloor }/\sqrt{n}$ has variance $\lfloor nt\rfloor /n$. Its linear interpolation isand ${\sup }_{0\le t\le 1}|Y_n(t)-R_n(t)|\le 1/\sqrt{n}$. Donsker's theorem says $Y_n\Rightarrow B$ in $C[0,1]$ with the uniform norm: this is convergence in distribution of random paths. The interpolation bound vanishes, but the theorem does not assert pointwise convergence of an arbitrarily fixed walk to an independently chosen Brownian motion. The usefulness is that many-step discrete motion has a universal continuous limit.^[ref]

A càdlàg modification of a stochastic process is a modification of a stochastic process whose paths are càdlàg on one event of probability one. Equality almost surely at every separately fixed time does not imply indistinguishability of stochastic processes when the original process lacks path regularity. A stochastic process starting at zero, with independent increments, stationary increments, and stochastic continuity, has a càdlàg modification and is a Lévy process under the intrinsic definition.

If two versions of a stochastic process both have càdlàg paths on an event of probability one, equality at every rational time holds simultaneously outside one null event. Approaching an arbitrary time by rationals from the right then gives equality at every time on the same event. Thus the stochastic processes are indistinguishable. Right continuity suffices; left limits are unnecessary for this conclusion.

A multiscale method for bounding the supremum of a stochastic process. Approximate each index by points ${\pi }_k(t)$ in successively finer metric nets, expand its difference from an anchor as a telescoping sum, and bound the largest increment at each scale. For a finite index set the finest net can be the entire set, making this decomposition exact. If $E_k$ is the set of approximation edges, then pointwise ${\sup }_t(X_t-X_{t_0})\le {\sum }_k{\max }_{(u,v)\in E_k}(X_u-X_v)$. Finite maxima controlled by moment-generating functions turn the sizes of the nets into metric entropy sums and integrals. The Dudley entropy integral and dyadic increment chaining are related instances.

For a Lévy process, stochastic continuity gives continuity of its characteristic function in time at every fixed frequency. The elementary estimate $|{\phi }_{X_s}(u)-{\phi }_{X_t}(u)|\le |u|\delta +2\mathbb{P}(|X_s-X_t|>\delta )$ proves this by first taking $s\to t$ and then $\delta \downarrow 0$. The argument works for any stochastically continuous stochastic process and needs no finite-moment assumption.

A continuous modification of a stochastic process is a modification of a stochastic process whose paths are continuous outside a single null event. For a process originally given only on a countable dense set, the analogous construction gives a continuous extension agreeing simultaneously at all original indices. The Kolmogorov continuity theorem and dyadic increment chaining are standard ways to obtain it.

A correlation time measures how rapidly a stationary stochastic process loses temporal dependence. Specify whether it means an exponential decay time, an integral of the normalized autocorrelation, or another width. For $R(t)=R(0)e^{-|t|/\tau }$, the exponential time and the one-sided integral ${\int }_0^{\infty }R(t)dt/R(0)$ both equal $\tau$.

For a real-valued stochastic process $(X_t)$ with finite second moments, write $m(t)=\mathbb{E}{X}_t$. Its covariance function isThe diagonal gives the variance at each time; off-diagonal values record linear dependence between times. For the independent standard normal random variables above, $C_X(m,n)=1_{\{m=n\}}$, whereas the shared Gaussian process $Y_n=Z$ has $C_Y(m,n)=1$ for all $m,n$. For a fair simple random walk on the integer line, $C(m,n)=\min (m,n)$; Brownian motion has the continuous-time counterpart $C(s,t)=\min (s,t)$. These formulas quantify how much random history the observations share.

Every covariance function is a positive-semidefinite kernel. Indeed, for any real coefficients $a_1,\dots ,a_k$,Thus a proposed covariance function must make every finite covariance matrix positive semidefinite. For a Gaussian process, the mean function and the covariance function determine every finite-dimensional distribution, including degenerate ones such as the shared process. Conversely, any real positive-semidefinite kernel satisfying $C(s,t)=C(t,s)$, together with a specified mean function, defines consistent multivariate normal distributions; the Kolmogorov extension theorem then constructs a Gaussian process. For a general stochastic process, covariance alone does not determine the full law. The exceptional completeness of this description is what makes Gaussian process regression possible with a mean and a covariance model.

A cross-spectrum relates the Fourier transforms of two random fields or two stochastic processes. The cross-spectrum of two stationary time series gives the discrete-time case.

For statistically homogeneous random fields, the three-dimensional cross-power spectrum is defined byunder the Fourier transform convention $U(\mathbf{x})=\int d^{3}kU(\mathbf{k})e^{i\mathbf{k}\cdot \mathbf{x}}/(2\pi {)}^3$. For isotropic fields it depends only on $k=|\mathbf{k}|$. A deterministic relation $V(\mathbf{k})=A(k)U(\mathbf{k})$ implies $P_{UV}=A^{\ast }{P}_{UU}$. An angular cross-power spectrum is defined analogously from spherical harmonic coefficients.

For a function on the dyadic rationals in $[0,1]$, let $A_n$ be its largest absolute adjacent increment on the level-$n$ grid. If $K_{\alpha }<\infty$ for $\alpha >0$, the function extends uniquely to a Hölder continuous function withChoose $n$ with $2^{-n}\le |t-s|<2^{1-n}$. The level-$n$ left approximations differ by at most two grid steps, and each finer approximation adds at most one level increment. Thus the difference is bounded by $2{\sum }_{j\ge n}{A}_j\le 2^{-n\alpha }{K}_{\alpha }$. For a stochastic process with $\Vert {\xi }_t-{\xi }_s{\Vert }_p\le C|t-s{|}^{\beta }$, the bound $\Vert A_n{\Vert }_p\le C{2}^{-n(\beta -1/p)}$ makes this series converge in the Lp norm for $0<\alpha <\beta -1/p$.

The entropy rate measures average new uncertainty per emitted symbol of a stochastic process. Predictability from earlier symbols can make this much smaller than the information entropy of one symbol. For example, a binary Markov source can emit a fair bit at every time while repeating its previous bit with probability $7/8$: each individual symbol has one bit of uncertainty, but the entropy rate is only about $0.544$ bits per symbol.

For a finite alphabet source $X=(X_1,X_2,\dots )$, definewhen the limit exists. Base-two logarithms give bits per symbol. Multiplying by a fixed emission frequency $f$ gives $fh(X)$ bits per unit time; the symbol rate and the uncertainty of the sequence are different inputs. A strictly stationary process on a finite alphabet always has an entropy rate, equal to its conditional entropy rate.^[ref]

For an independent and identically distributed source, the chain rule for information entropy gives $H(X_1,\dots ,X_n)=nH(X_1)$, hence $h(X)=H(X_1)$. With dependence, conditioning reduces entropy and makes the past useful. The examples under Markov information source show this benefit using exact finite-block calculations.