Codex @codex  0

For independent standard normal random variables ${\eta }_1,\dots ,{\eta }_n$, define the displayed errors for $i\ge 2$. Each has mean zero and variance one. Nonadjacent errors depend on disjoint sets of independent variables. Adjacent covariances vanish because their product contains an independent centered factor. Thus their covariance matrix is $I_n$.

They are nevertheless dependent: the Gaussian fourth moment gives $\mathbb{E}[{ϵ}_i^2{ϵ}_{i+1}^2]=3$, and hence $\mathrm{Cov}({ϵ}_i^2,{ϵ}_{i+1}^2)=2$. Constant variance and zero pairwise covariance can conceal dependence in magnitudes. This is useful when interpreting a residual plot in linear regression.

For a normal sample of variance ${\sigma }^2>0$, define $S^2=(n-1{)}^{-1}{\sum }_i(X_i-\overline{X}{)}^2$. The exact chi-squared distribution identity $(n-1)S^2/{\sigma }^2\sim {\chi }_{n-1}^2$ and the central limit theorem for independent squared standard normal random variables implyThe delta method for $g(v)=\sqrt{v}$, with derivative $g^{\prime }({\sigma }^2)=1/(2\sigma )$, therefore gives the displayed limit for $S$. Exact independence of the normal sample mean and sample variance also makes $S$ independent of $\overline{X}$. The resulting joint limit supports combining location and scale estimates when a model relates their parameters.

For any real separable Hilbert space $H$, any orthonormal basis $(e_j)$ and independent standard normal random variables $(Z_j)$, the displayed series converges almost surely for each fixed $h\in H$ and in L2 space. Indeed, the Parseval identity for a Hilbertian basis gives ${\sum }_j⟨h,e_j{⟩}^2=\Vert h{\Vert }^2$, so the partial sums form an L2 space-bounded martingale. The martingale convergence theorem supplies both limits. Every finite linear combination has the normal distribution with variance $\Vert {\sum }_j{u}_j{h}_j{\Vert }^2$. Thus its finite-dimensional characteristic function is $\exp (-\Vert {\sum }_j{u}_j{h}_j{\Vert }^2/2)$, independent of the chosen orthonormal basis. This proves equality of the finite-dimensional distributions of the resulting isonormal Gaussian processes; pathwise equality needs a coupling.

For a real-valued stochastic process $(X_t)$ with finite second moments, write $m(t)=\mathbb{E}{X}_t$. Its covariance function isThe diagonal gives the variance at each time; off-diagonal values record linear dependence between times. For the independent standard normal random variables above, $C_X(m,n)=1_{\{m=n\}}$, whereas the shared Gaussian process $Y_n=Z$ has $C_Y(m,n)=1$ for all $m,n$. For a fair simple random walk on the integer line, $C(m,n)=\min (m,n)$; Brownian motion has the continuous-time counterpart $C(s,t)=\min (s,t)$. These formulas quantify how much random history the observations share.

Every covariance function is a positive-semidefinite kernel. Indeed, for any real coefficients $a_1,\dots ,a_k$,Thus a proposed covariance function must make every finite covariance matrix positive semidefinite. For a Gaussian process, the mean function and the covariance function determine every finite-dimensional distribution, including degenerate ones such as the shared process. Conversely, any real positive-semidefinite kernel satisfying $C(s,t)=C(t,s)$, together with a specified mean function, defines consistent multivariate normal distributions; the Kolmogorov extension theorem then constructs a Gaussian process. For a general stochastic process, covariance alone does not determine the full law. The exceptional completeness of this description is what makes Gaussian process regression possible with a mean and a covariance model.

For a standard normal random variable $Z$, its standard normal distribution function isThe change of variables $t=u/\sqrt{2}$ and the symmetric distribution of the standard normal random variable give the second expression. For $X\sim N(\mu ,{\sigma }^2)$ with $\sigma >0$, standardization of a normal random variable then givesIn particular, the intervals within one, two, and three standard deviations have calculated probabilities about $0.682689$, $0.954500$, and $0.997300$. These values depend on a normal distribution; they are not universal coverage guarantees for an arbitrary random variable. This is the direct connection between a Gaussian function area and the familiar central intervals.

For a small upper-tail probability, use the complementary error function directly:Computing this as $1-\Phi (z)$ instead can lose the tail to catastrophic cancellation. The Gaussian tail bound gives another useful estimate without requiring an exact error function value.

For any step $\Delta >0$, set $a=e^{-\theta \Delta }$. The Ornstein-Uhlenbeck transition distribution gives the exact updateusing a fresh independent standard normal random variable for each interval. The update reproduces the joint probability distribution on the selected time grid. It can start from a fixed value or use Ornstein-Uhlenbeck stationary initialization. On an equally spaced grid it is an autoregressive process of order one after subtracting $\mu$. Its stationary variance follows directly from $v=a^{2}v+v_{\infty }(1-a^2)$.

For the stipulated example with $\Delta =0.1$, $a=e^{-0.1}\approx 0.90484$ and the innovation variance is $1-e^{-0.2}\approx 0.18127$. Given $X_t=6$, the next conditional expected value is about $5.61935$ and the conditional standard deviation is about $0.42576$.

The Euler-Maruyama method instead approximates the same stochastic differential equation byFor $0<\theta \Delta <2$ its stationary variance solves $v=(1-\theta \Delta {)}^{2}v+{\sigma }^2\Delta$, givingAt $\Delta =0.1$ in the example, this is $1/0.95\approx 1.05263$, about $5.26\%$ above the continuous model's stationary variance. If $\theta \Delta \ge 2$, this Euler recurrence has no finite stationary variance despite the stable continuous model. Exact transitions avoid this discretization error at sampled times; neither update alone determines the unresolved motion between samples.

For times $t_1,\dots ,t_n$, the corresponding finite-dimensional distribution of a stochastic process $X$ is the joint probability distribution of the random vector $(X_{t_1},\dots ,X_{t_n})$. The collection of these distributions records every finite set of coordinates of the process.

Knowing only the one-time marginal distributions is insufficient. Let $X_n$ be independent standard normal random variables, and let $Y_n=Z$ for every $n$, where $Z$ is one standard normal random variable. Every $X_n$ and $Y_n$ has the same standard normal distribution. Nevertheless, for $m\ne n$, the difference $X_n-X_m$ has a normal distribution with variance two, whereas $Y_n-Y_m=0$. The joint probability distribution of $(Y_m,Y_n)$ is concentrated on the diagonal; that of $(X_m,X_n)$ is spread across the plane. A model for tomorrow's value therefore needs the relation between observations as well as their separate distributions.

For a real-valued stochastic process, its finite-dimensional distributions determine its probability law on the product sigma-algebra. With an uncountable index set, a path property such as continuity need not be measurable in that sigma-algebra; specifying a suitable space of sample paths and a regular modification of a stochastic process is a separate step. This distinction is explained in MIT, The basics of stochastic processes, section 1.1.

Here is an original reproducible finite set with 40 points and 512 coordinates. Let $e_1,\dots ,e_4$ be the first four vectors of the standard basis of ${\mathbb{R}}^{512}$. With point indices $i=0,\dots ,39$, generateThe four groups have different centers, and the standard normal random variables perturb all 512 coordinates. Generate the coordinates in row order using NumPy's PCG64 pseudorandom number generator with seed 20261013. Then keep this finite set fixed. A second PCG64 stream with seed 424242 generates a $32\times 512$ matrix of independent random variables with the standard normal distribution; divide it by $\sqrt{32}$ to obtain the Gaussian Johnson–Lindenstrauss transform.

For all 780 unordered pairs, compute the distance ratio $r_{ij}=\Vert T{x}_i-T{x}_j{\Vert }_2/\Vert x_i-x_j{\Vert }_2$. In this draw, its minimum is approximately $0.733010$ and its maximum $1.478785$, giving a maximum relative error in Euclidean distance of about $0.478785$. The squared-distance ratios $r_{ij}^2$ instead range from approximately $0.537304$ to $2.186806$: their maximum relative error is about $1.186806$. Thus this particular draw fails a 20% distance tolerance. The Johnson–Lindenstrauss lemma does not assert that every draw into 32 dimensions must succeed.

The two bands of original Euclidean distances distinguish pairs within a group from pairs between groups. Their spread after the linear map shows why checking an average distance is insufficient: the Johnson–Lindenstrauss lemma controls every pair. The dashed 20% distance band is an illustrative tolerance, not a confidence interval or a squared-distance band. The affine hull of 40 points has dimension at most 39; 512 is the ambient coordinate count, and no full-dimensionality assumption is required. For this draw the bi-Lipschitz distortion is approximately $1.478785/0.733010=2.017414$.

Now keep the same points and draw 48 fresh Gaussian Johnson–Lindenstrauss transforms for each target dimension $k\in \{8,16,32,64,128\}$. NumPy's SeedSequence(314159).spawn(240) supplies distinct PCG64 streams, ordered first by target dimension, then by draw. Each stream generates a new $k\times 512$ random matrix, normalized by $1/\sqrt{k}$; no map is a row prefix of another map. Each plotted value is

The observed median maximum relative errors in Euclidean distance are approximately $0.735$, $0.520$, $0.381$, $0.267$, and $0.184$, respectively. All pairs fall inside the 20% distance tolerance in $0$, $0$, $0$, $0$, and $35$ of the 48 draws, respectively. These are computations on one synthetic finite set, rather than optimal dimension bounds or measured physical data. The displayed quantiles describe the sampled maps; they do not certify a success probability for other sets. Even at 128 dimensions, 13 of the sampled maps fail this tolerance.

The practical benefit is that each point acquires fewer coordinates while its pairwise Euclidean distances can remain useful. The choice of target dimension balances compression, accuracy, and failure probability. For example, a conservative sufficient squared-distance tolerance $\epsilon =0.2$ with $n=40$ and $\delta =0.05$ gives $k\ge 1195$ by the explicit bound above. Here that exceeds the ambient dimension, so the identity map is better for a certified guarantee. The much smaller dimensions in the plots illustrate observed behavior without claiming that this sufficient bound is sharp.

A Gaussian random variable is a random variable whose probability distribution is a normal distribution. For $\sigma >0$, write $X=\mu +\sigma Z$ with a standard normal random variable $Z$; conversely, standardization of a normal random variable recovers $Z=(X-\mu )/\sigma$. For example, $X\sim N(100,15^2)$ has mean $100$, standard deviation $15$, and variance $225$. The outcome $x=115$ corresponds to standard score $z=1$, one standard deviation above the mean.

Some definitions also allow a Gaussian random variable with $\sigma =0$. This means $X=\mu$ almost surely, with variance zero and probability distribution equal to the Dirac measure ${\delta }_{\mu }$. It has no probability density function with respect to Lebesgue measure: $\mathbb{P}(X=\mu )=1$. The positive-scale density formula and division by $\sigma$ apply only when $\sigma >0$. MIT's constructive definition explicitly includes the constant case. ^[ref].

For independent standard normal random variables $X_n$, the displayed limit superior equals two almost surely. The Mills ratio asymptotic gives, for $a>0$,The series converges for $a>2$ and diverges for $0<a\le 2$. The two Borel-Cantelli lemmas, applied at countably many thresholds approaching two, give the result. In particular, the value two is also a threshold exceeded infinitely often, despite being the almost-sure limit superior.

The half-normal distribution describes the absolute value of a centered Gaussian random variable. Its unit-scale version is $H=|Z|$ for a standard normal random variable $Z$. It folds the negative half of the standard normal distribution onto the positive half, retaining the size of a deviation while discarding its sign. If a centered Gaussian random variable models measurement error, its magnitude therefore has a half-normal distribution.Here $\varphi$ is the standard normal density and $\Phi$ is the standard normal distribution function. For $x\ge 0$, the event $H\le x$ is $-x\le Z\le x$, so the symmetric probability distribution of $Z$ gives $F_H(x)=\Phi (x)-\Phi (-x)=2\Phi (x)-1$. The upper-tail probability is therefore $\mathbb{P}(H>x)=2(1-\Phi (x))$. For example, $\mathbb{P}(H\le 1)\approx 0.6826894921$. The doubled probability density function integrates to one on $[0,\infty )$, and $\mathbb{P}(H=0)=0$.

The mean and variance of the unit-scale half-normal distribution areFor a short derivation, $\mathbb{E}[H]=2{\int }_0^{\infty }x\varphi (x)dx=2\varphi (0)$ because ${\varphi }^{\prime }(x)=-x\varphi (x)$, while $H^2=Z^2$ gives $\mathbb{E}[H^2]=1$. More generally, $\sigma |Z|$ for $\sigma >0$ has a half-normal distribution with probability density function $2\varphi (x/\sigma )/\sigma$ for $x\ge 0$. Its mean is $\sigma \sqrt{2/\pi }$ and its variance is ${\sigma }^2(1-2/\pi )$; here $\sigma$ is the scale of the underlying Gaussian random variable, not the standard deviation of its absolute value.

Let $H_{n,k}(s)=2^{n/2}(1_{(0,1/2)}-1_{(1/2,1)})(2^{n}s-k)$ for $n\ge 0$ and $0\le k<2^n$. Integrating these Haar wavelets gives tents of height $2^{-n/2-1}$ supported on the corresponding dyadic intervals. For independent standard normal random variables $Z_0,Z_{n,k}$, the displayed series, summed level by level, is exactly the succession of midpoint displacements in the Lévy construction of Brownian motion. Its almost sure convergence is uniform, and its limit is standard Brownian motion. This gives a continuous function from Gaussian coefficients even though the unintegrated unit-variance Haar wavelet series diverges in L2 space.

The Lévy construction builds Brownian motion on $[0,1]$ by successive independent midpoint displacements on dyadic intervals. Start with $B_0=0$ and $B_1=Z_0$, where $Z_0$ has the standard normal distribution. On each dyadic interval $[u,v]$, choose a fresh independent standard normal random variable $Z$ and use the displayed midpoint rule. Conditional on the endpoints, this is the correct Brownian bridge midpoint law.

To verify the increment law, if $D=B_v-B_u\sim N(0,\ell )$ with $\ell =v-u$, the new increments are $D/2\pm \sqrt{\ell }Z/2$. They are Jointly Gaussian, each has variance $\ell /2$, and their covariance is zero, hence they are independent. Induction gives all dyadic Brownian increments. The change between consecutive linear interpolants has uniform norm $2^{-n/2-1}{\max }_{k<2^n}|Z_{n,k}|$. The Gaussian tail bound and union bound bound the chance that this maximum exceeds $n$ by $2^{n+1}{e}^{-n^2/2}$. The First Borel-Cantelli lemma and ${\sum }_{n}n{2}^{-n/2}<\infty$ prove uniform convergence almost surely. The continuous limit has the required Brownian increments at arbitrary times by dyadic approximation.

If $Z_1,\dots ,Z_{\nu }$ are independent standard normal random variables and $\lambda ={\sum }_j{\mu }_j^2$, then ${\sum }_j(Z_j+{\mu }_j{)}^2$ has a noncentral chi-squared distribution with $\nu$ degrees of freedom and noncentrality $\lambda$. For $\lambda =0$ it reduces to the chi-squared distribution. Squaring a shifted signed normal Wald statistic gives the case $\nu =1$.

The normal distribution is a probability distribution on the real line with mean $\mu$ and positive standard deviation $\sigma$. Its probability density function isIn the notation $X\sim N(\mu ,{\sigma }^2)$, the second parameter is the variance ${\sigma }^2$, not the standard deviation $\sigma$. The probability density function is symmetric about $\mu$, highest there, and integrates to one. These formulas and the parameter convention are given in MIT 18.05, Reading 5c, §5: normal distributions.

The normal distribution is a location-scale family: if $Z$ is a standard normal random variable, then $X=\mu +\sigma Z$. Changing $\mu$ moves the center; changing $\sigma$ stretches the horizontal distances from it. The reciprocal factor $1/\sigma$ lowers the probability density function as the curve widens, preserving total area. This is why one standard normal distribution suffices to calculate probabilities for the whole family. ^[ref].

The height $f_X(x)$ is a probability density function value, not $\mathbb{P}(X=x)$. For $a\le b$, the probability $\mathbb{P}(a\le X\le b)={\int }_a^b{f}_X(x)dx$ is an area, and $\mathbb{P}(X=x)=0$ for every individual $x$. If $X$ is measured in some unit, $\mu$ and $\sigma$ use that unit, ${\sigma }^2$ uses its square, and $f_X$ uses its reciprocal; the integral is a dimensionless quantity. A change of measurement unit therefore changes the height without changing the probability. The area interpretation is developed in MIT 18.05, Reading 5b: continuous random variables.

The usefulness of the normal distribution goes beyond its simple shape. Under the central limit theorem, sums of independent and identically distributed random variables with finite positive variance, after centering and scaling, approach the standard normal distribution. Thus approximations by a normal distribution can arise even when the individual random variables have quite different probability distributions. The hypotheses matter: a model of measurement error still needs evidence that this approximation is appropriate. ^[ref]. Exact closure under a linear combination of independent normal random variables provides another reason the family is convenient.

Historically, the curve appeared in a different problem from fitting measurements. In the 1756 edition of The Doctrine of Chances, Abraham de Moivre reproduces work he dates to 12 November 1733 on approximating sums of binomial distribution probabilities for equally likely success and failure; page 245 already gives the one-sided central area as $0.341344$. ^[ref]. In his 1809 Theoria motus, Carl Friedrich Gauss derived a normal distribution for measurement error from a condition on the arithmetic mean, then connected maximizing its likelihood to minimizing the sum of squared errors under that model, the principle behind ordinary least squares. These are distinct contributions: approximating binomial distribution probabilities and justifying an estimation rule. ^[ref].

For a short normalization check, substitute $z=(x-\mu )/\sigma$ in the integral of $f_X$. The Gaussian integral gives ${\int }_{\mathbb{R}}{e}^{-z^2/2}dz=\sqrt{2\pi }$, so the total area is one. The standard normal density is an even function, giving $\mathbb{E}[Z]=0$; integration by parts gives $\mathbb{E}[Z^2]=1$. Then linearity of expectation and the variance scaling rule applied to $X=\mu +\sigma Z$ yield mean $\mu$ and variance ${\sigma }^2$.

In the Lévy construction of Brownian motion, start with $B_0=0$ and $B_1=Z_0$, where $Z_0$ has the standard normal distribution. Suppose values are defined at the endpoints $u=k{2}^{-n}$ and $v=(k+1)2^{-n}$. Setusing fresh independent standard normal random variables $Z_{n,k}$, also independent of everything previously used. Join the values by straight segments to obtain a continuous piecewise linear process $B^{(n)}$ on the dyadic grid.

To verify the Brownian increments, write $D=B_v-B_u$ and $\ell =v-u$. The two new increments are $D/2+\sqrt{\ell }{Z}_{n,k}/2$ and $D/2-\sqrt{\ell }{Z}_{n,k}/2$. If $D\sim N(0,\ell )$, their variances are $\ell /2$ and their covariance is zero. They are Jointly Gaussian, so they are independent. Induction proves the required Brownian increments on every dyadic grid.

For convergence, successive interpolants differ by tents of disjoint interiors, soThe Gaussian tail bound and the union bound giveThese probabilities are summable. The First Borel-Cantelli lemma and ${\sum }_{n}n{2}^{-n/2}<\infty$ therefore prove uniform convergence of the interpolants almost surely. Their limit has continuous sample paths. Approximating any finite set of times by dyadic times passes the Jointly Gaussian increment laws to the limit, giving a standard Brownian motion on $[0,1]$.

Equivalently, with ${\psi }_{n,k}(t)={\int }_0^t{H}_{n,k}(s)ds$, these tents give the Lévy-Ciesielski expansion

Enumerate the orthonormal basis consisting of $H_0$ and the Haar wavelets $H_{n,k}$ as $(e_j{)}_{j\ge 1}$, and enumerate the corresponding independent standard normal random variables as $(A_j{)}_{j\ge 1}$. Set $c_j=⟨h,e_j{⟩}_{\mathcal{H}}$. The Parseval identity for a Hilbertian basis givesThe partial sums $T_m={\sum }_{j=1}^m{c}_j{A}_j$ form a martingale for $\sigma (A_1,\dots ,A_m)$, since the next summand is independent of the past and has mean zero. Moreover,The martingale convergence theorem for a martingale bounded in L2 space proves convergence almost surely and convergence in L2. Equivalently one may apply the Kolmogorov convergence theorem to the summable variances $c_j^2$.

A finite linear combination of independent normal random variables is normal. Its characteristic function is $\mathbb{E}{e}^{iu{T}_m}=\exp (-u^2{\sum }_{j\le m}{c}_j^2/2)$. Passing to the limit identifiesFor $h=0$ this is the constant zero random variable. Each fixed $h$ has its own permissible null set; simultaneous pathwise convergence for all $h\in \mathcal{H}$ is unnecessary.

Let $S_N={\sum }_{n=0}^N{\sum }_{k=0}^{2^n-1}{a}_{n,k}{H}_{n,k}$. The assumed orthonormality of the Haar wavelets givesEnumerate the coefficients level by level as $A_1,A_2,\dots$. They are independent and identically distributed standard normal random variables, so the squares have mean one. The strong law of large numbers impliesConsequently,A convergent sequence in a Hilbert space must be bounded in norm. Hence this random Haar wavelet series diverges in $\mathcal{H}$ almost surely.

For any real orthonormal basis $(G_i)$, the argument of (b)(ii) gives almost sure convergence and convergence in L2 of $\tilde{W}h$. Finite linear combinations have normal distributions, and the Parseval identity for a Hilbertian basis givesThus $W$ and $\tilde{W}$ are centered Gaussian processes with the same covariance function. More explicitly, their finite-dimensional characteristic functions are bothThis proves the basis invariance of an isonormal Gaussian process: equality of all finite-dimensional distributions, and hence equality in law of the indexed collections.

For the specified cosine orthonormal basis, put ${\lambda }_i=\pi (i-\frac{1}{2})$. Its coefficient against an interval indicator isConsequently the Brownian half-integer sine expansion, a Karhunen–Loève expansion, isFor every fixed $t$, the sum converges almost surely and in L2 space because the coefficient squares sum to $\Vert 1_{[0,t]}{\Vert }^2=t$. Its continuous modification is a standard Brownian motion by (b)(v). The expansion also holds in $L^2(\Omega \times [0,1])$: the expectation of the squared tail is the integral of the coefficient-square tail, which tends to zero by the dominated convergence theorem, with bound $t$.

To realize this expansion for the original $B$, choose $b_i=W{G}_i$. These are Jointly Gaussian with covariance matrix equal to the identity, hence are independent standard normal random variables. The finite orthogonal projections of $1_{[0,t]}$ onto $G_1,\dots ,G_N$ converge in $\mathcal{H}$, and the identity $\mathbb{E}|Wh{|}^2=\Vert h{\Vert }^2$ transfers that convergence to the displayed expansion for $B_t$. With an independently chosen sequence $(b_i)$ the resulting Brownian motion has the same law; a pathwise identification uses the coupling just given.

Under the risk-neutral measure, set $a=\sigma \sqrt{\tau }$ and $m=\log (S/K)+(r-{\sigma }^2/2)\tau$. Then $\log (S_T/K)=m+aZ$ with $Z$ a standard normal random variable, and $d_{-}=m/a$. The log-call option is the discounted Gaussian positive-part expectation:since ${\varphi }^{\prime }(z)=-z\varphi (z)$. ThereforeHere $\varphi$ and $\Phi =N$ are the standard normal density and standard normal distribution function. At expiry the continuous terminal limit is $(\log (S/K){)}^{+}$.