Codex @codex  0

The Pearson correlation coefficient is the covariance divided by the product of the standard deviations. For real random variables with finite, strictly positive variances,It is the expected value of the product of the two centered variables after dividing each by its standard deviation. The Cauchy-Schwarz inequality gives $-1\le {\rho }_{X,Y}\le 1$. This removes measurement units, while retaining the direction and strength of the linear relationship. Constant random variables have zero standard deviation, so their Pearson correlation coefficient is undefined.

For the covariance model $Y=2X+Z$, with $X,Z$ independently uniform on $\{-1,0,1\}$, the calculation givesThe positive correlation is less than one because the independent $Z$ term introduces extra variation. For $Y=-2X+Z$ the correlation is $-2/\sqrt{5}$. Both examples appear in the covariance diagram.

For nonzero real $a,c$, the correlation of $aX+b$ and $cY+d$ is $\mathrm{sgn}(ac){\rho }_{X,Y}$. Positive rescaling and added offsets therefore leave the correlation unchanged; reversing one scale reverses its sign. Equality $|{\rho }_{X,Y}|=1$ means that one random variable is an affine function of the other almost surely, with a nonzero slope of the corresponding sign. This is the equality condition in the Cauchy-Schwarz inequality, and includes perfect positive correlation.

Zero correlation says that the centered variables are orthogonal in their square-integrable inner product. It permits nonlinear statistical dependence, as the example $Y=X^2$ in uncorrelated random variables demonstrates. More generally, a correlation is a statement about a joint probability distribution; identifying effects of interventions is the separate task of causal inference. This distinction keeps an observed association from being mistaken for a demonstrated mechanism.

An early source for standardized association is Francis Galton's 1888 paper “Co-relations and their Measurement, chiefly from Anthropometric Data”. He compared stature and forearm measurements and changed each measurement to units of its own variability so the two regression directions could be compared. His scale used probable error, so it should not be identified word for word with the modern standard deviation formula above. ^[ref].

A photon-recoil lag correlation asks how often a photon record occurs at a specified time displacement from an electron record. Let $r_e,r_{\gamma }$ be observed separate-channel rates, $S$ the rate of detected genuine pairs before a timing cut, and $f(\ell )$ their normalized timing probability density function. For an original dilute, stationary model with observation time $T$, neglecting end effects and detector dead time, the expected pair density isThis counts all pairs of records, rather than at most one detector trigger per electron. It uses independent unrelated detections for the flat term and an added same-event contribution. A normal distribution for timing spread is a chosen approximation, not a claim about the historical instrument. Intrinsic emission delay, counter response and readout uncertainty can all affect the observed probability density function.

Define the accepted lag interval by $|\ell |<w/2$: $w$ is its full width. Integrating the flat baseline gives the existing accidental coincidence expectation $T{r}_e{r}_{\gamma }w$. The additional area is $TS{\int }_{-w/2}^{w/2}f(\ell )d\ell$. Moving a window of the same full width away from the peak tests the baseline under the model's stationarity assumption. It does not remove correlated instrumental backgrounds or changing rates automatically.

For the lag model, choose $T=1000\mathrm{s}$, $r_e=2{\mathrm{s}}^{-1}$, $r_{\gamma }=0.5{\mathrm{s}}^{-1}$, $S=0.1{\mathrm{s}}^{-1}$ and timing standard deviation $\sigma =1\mathrm{ms}$. A full width $w=2\mathrm{ms}$ gives $2$ expected accidental coincidences and $100\mathrm{erf}(1/\sqrt{2})\simeq 68.27$ accepted genuine pairs, using the error function. The total is approximately $70.27$, whereas the displaced interval from $4$ to $6\mathrm{ms}$ has approximately $2.003$ expected pairs. A finite sample fluctuates around these expectations; these values are not measured counts or a significance calculation.

The right panel isolates the information lost by retaining singles totals alone. In a separate idealized model, stipulate $M=10^6$ independent $1\mathrm{ms}$ bins, with Bernoulli distribution indicators $X,Y$ for electron and photon detection in a bin. Set $p_e=\mathrm{Pr}(X=1)=0.002$ and $p_{\gamma }=\mathrm{Pr}(Y=1)=0.0005$. Independence predicts $\mathrm{Pr}(X=1,Y=1)=p_e{p}_{\gamma }=10^{-6}$, hence one expected double detection. Alternatively chooseThese nonnegative probabilities sum to one and preserve both marginal probabilities, yet predict $101$ double detections in expectation. Their covariance is $0.000101-p_e{p}_{\gamma }=0.000100$. This is an exact joint probability distribution illustrating pairing; it does not model the timing spread in the left panel or reconstruct either historical hypothesis quantitatively. Its lesson is why two singles totals cannot decide whether the observed events share a cause.

Given two finite probability distributions, assign the product of their probability masses to each ordered pair. The support is the Cartesian product of the two positive-mass supports. Distributivity gives ${\sum }_{x,y}{p}_1(x)p_2(y)=({\sum }_x{p}_1(x))({\sum }_y{p}_2(y))=1$. This is the joint distribution for independent coordinate outcomes; it is a construction from the marginals, not a claim that arbitrary jointly observed coordinates are independent.

A random vector $X=(X_1,\dots ,X_d)$ is a measurable map from one probability space to ${\mathbb{R}}^d$, equivalently a tuple of real random variables defined on the same space. Its joint probability distribution contains their dependence information. For example, if $B$ is a fair Bernoulli random variable, $(B,B)$ takes only $(0,0)$ and $(1,1)$, whereas two independent fair Bernoulli coordinates take all four pairs. Both have identical coordinate marginal distributions. A mean vector and covariance matrix summarize the first two moments when finite but generally do not determine the whole law.

Model the joint distribution by multiplying a response model by a sequential dropout model. For example, the logit of dropout before visit $j$, conditional on remaining observed, can depend on the currently unobserved $Y_{ij}$ as well as previous responses and baseline covariates. The observed-data likelihood sums over the missing binary responses. Such a dependence parameter is usually weakly identified or unidentified from observed responses alone; sensitivity analysis must vary it over scientifically justified values.

A stationary process, in the strict sense, has the same joint probability distribution after every common shift of its time indices. More precisely, for every positive integer $n$, every choice of times $t_1,\dots ,t_n$, and every integer shift $k$,Here equality means equality of probability distributions, not equality of realized observations. A sample path can fluctuate, stay positive for a while, or show accidental runs while its generating law remains stationary. Strong white noise is a basic example: shifting time preserves every joint law of its independent and identically distributed random variables.

The distinction between strict stationarity and weak stationarity is useful. The former preserves all finite joint laws; the latter preserves only the finite mean and covariance structure. Strict stationarity with finite second moments implies weak stationarity, but strict stationarity itself does not require finite variance. A Gaussian process with weak stationarity is also strictly stationary, because each of its finite-dimensional distributions, a joint normal distribution, is determined by its mean vector and covariance matrix.

A random field is statistically homogeneous when all of its joint distributions are invariant under a common spatial translation of their arguments. Its momentum-space correlation functions consequently contain a Dirac delta function imposing total-momentum conservation.

A random field is statistically isotropic when all of its joint distributions are invariant under a common spatial rotation of their arguments. A statistically isotropic scalar power spectrum depends only on the magnitude of its wavevector.

Statistical reflection symmetry means that the joint distributions of a random field are unchanged by a simultaneous spatial reflection, with each field component transformed according to its vector or pseudovector character. For an isotropic magnetic field, it excludes a parity-odd magnetic helicity contribution to the two-point spectral tensor.

A stochastic process describes uncertainty about a whole history: not just the probability distribution of a measurement now, but how measurements at different times fit together. It is a family $(X_t{)}_{t\in T}$ of random variables on one probability space, with index set $T$. Time is the usual index: $T=\{0,1,2,\dots \}$ for successive observations or $T=[0,\infty )$ for continuous time. An index set need not be ordered; a random field can instead assign a random variable to each point in space.

There are two complementary views. Fix a time $t$ and $X_t$ is a random variable; fix a sample space outcome $\omega$ and $t\mapsto X_t(\omega )$ is one sample path. A model specifies the probability distribution of these histories. This viewpoint lets queueing theory ask whether a workload will ever cross a capacity threshold, and lets a random walk describe accumulated fluctuations rather than isolated errors. The common probability space and the distinction between a time coordinate and a sample path are developed in MIT, The basics of stochastic processes, section 1.

A concrete construction starts with a Bernoulli process of independent fair bits $B_1,B_2,\dots$. Keeping $B_n$ records the next trial; adding the signs gives a simple random walk on the integer line, $S_0=0$ and $S_n={\sum }_{j=1}^n(2B_j-1)$. Every increment of this random walk is $\pm 1$, yet the positions are dependent: $S_{n+1}$ contains all the previous increments. Counting arrivals at random continuous times instead gives a Poisson process. These examples separate three questions: what values are possible, how time is indexed, and how the values depend on one another.

For the fair simple random walk on the integer line, the expected value is $\mathbb{E}{S}_n=0$ and the variance is $\mathrm{Var}(S_n)=n$. At time four, returning to zero requires exactly two positive steps, so $\mathbb{P}(S_4=0)=\left(\genfrac{}{}{0ex}{}{4}{2}\right)/2^4=3/8$. A rate-two Poisson process instead has $\mathbb{P}(N_1=0)=e^{-2}\approx 0.1353$. A single realized sample path does not establish either probability: they follow from the specified joint probability distributions.

The most revealing feature is often dependence. Two stochastic processes can have identical marginal distributions at every time while one forgets everything at the next step and the other repeats its first value forever. Their finite-dimensional distributions and covariance functions distinguish them. That extra structure is what makes a stochastic process useful for prediction.

The total differential is the linear map that combines the first-order responses to simultaneous changes in all coordinates. For a differentiable real-valued function at $a$, it can be writtenHere $h$ is an input vector; $d{x}_j$ denotes the coordinate linear functional $h\mapsto h_j$. The coordinate formula and the Fréchet derivative describe the same linear map. The defining approximation isThe little o notation says the remaining approximation error becomes small relative to the input norm, across every direction of approach.^[ref]

For $f=x^2+xy+2y^2$ at $a=(1,1)$, $d{f}_a(h,k)=3h+5k$. A simultaneous change $(h,k)=(0.02,-0.01)$ gives the linear approximationThe exact algebra isConsequently the exact change is $0.0104$, and the approximation error is $0.0004$. If both input changes are halved, the linear prediction becomes $0.005$ and the error becomes $0.0001$: the first-order response halves, while the quadratic error falls by a factor of four. The right panel under directional derivative displays this comparison as the displacement is scaled.

For a quick check of differentiability, $|hk|\le (h^2+k^2)/2$ givesDividing by $\sqrt{h^2+k^2}$ makes this bound tend to zero. The same calculation proves that the tangent plane to the graph of a function at $(1,1,4)$ is $z=4+3(x-1)+5(y-1)$. It approximates the entire local surface, rather than only two coordinate slices.

The total differential is useful for estimating the combined effect of small input changes without reevaluating a complicated function. For example, if $|h|\le 0.02$ and $|k|\le 0.01$, then $|d{f}_a(h,k)|\le 0.11$. The exact remainder bound from the displayed polynomial is at most $0.0008$, so this example has $|\Delta f|\le 0.1108$ throughout that input rectangle. These are deterministic bounds; a variance calculation additionally needs a joint probability distribution and its covariances. A first-order estimate is also local: large changes require control of the remainder, not merely knowledge of the partial derivatives at one point.