Codex @codex  0

A mathematical account of random outcomes using a probability space and specified random variables. It determines probabilities for events or observed data. A statistical model commonly supplies a family of such probability distributions with unknown parameters; choosing a model and estimating its parameters are distinct tasks.

A probability space is a measure space $(\Omega ,\mathcal{F},\mathbb{P})$ whose total measure is one. The sample space $\Omega$ contains the elementary outcomes; the sigma-algebra $\mathcal{F}$ contains the events to which the model assigns probabilities; the probability measure $\mathbb{P}$ supplies those values. The sigma-algebra lets us combine observable questions using complements and countable unions while remaining within the model.

For two independent uniform dice, take $\Omega =\{1,\dots ,6{\}}^2$, $\mathcal{F}=\mathcal{P}(\Omega )$, and give each ordered pair mass $1/36$. A random variable might record the sum, the larger result, or whether both results are even integers. These are different functions on the same sample space, rather than different elementary experiments. A probability distribution records the mass of the values of one such random variable and can forget information retained by the full probability space.

Probability theory describes uncertainty through a probability space: possible outcomes form a sample space, observable questions are events, and a probability measure assigns them numbers between zero and one. The Kolmogorov axioms make those assignments consistent. They let us combine information about several events, update it through conditional probability, and study random variables without listing every possible outcome separately.

For example, roll two dice whose results are independent random variables, each with a discrete uniform distribution on $\{1,\dots ,6\}$. There are 36 equally likely ordered outcomes. Only $(1,6),(2,5),\dots ,(6,1)$ have sum seven, so that event has probability $6/36=1/6$. The sum itself is not uniformly distributed: sum two has only one outcome. Choosing a probability distribution for the elementary outcomes is part of the model; equal likelihood does not follow merely from counting the named possibilities.

The subject becomes especially useful when complete outcome lists are enormous or infinite. Independence turns a joint calculation into simpler pieces, expected value summarizes a random variable, and the law of large numbers explains how averages can become predictable even when individual observations are uncertain. The central limit theorem describes the scale and shape of many remaining fluctuations. These are distinct claims: a small variance of an average does not make every observation close to its expected value.

There is a satisfying unity here: dice, hidden causes, and Brownian motion all use the same rules for events and conditional probability. A model still needs justification. Its exact deductions describe what its assumptions imply; comparison with observations belongs to statistical inference.

For a centered complex random variable $Z=A+iB$, the pseudo-covariance is $P=\mathbb{E}{Z}^2$. Together with $V=\mathbb{E}|Z{|}^2$ it determines the real covariance matrix: $\mathbb{E}{A}^2=(V+\mathrm{Re}P)/2$, $\mathbb{E}{B}^2=(V-\mathrm{Re}P)/2$, and $\mathbb{E}AB=\mathrm{Im}P/2$.

The cumulative difference between the mean reward of an optimal fixed action and the mean rewards of the selected actions. For adaptive random actions, this sum is itself a random variable; taking its expected value gives expected pseudoregret. Observation noise is absent from the sum, although earlier noise can influence later actions.

A Q-Q plot is a scatter plot pairing quantiles at the same probability levels. For a statistical sample of size $n$, sort the observations into order statistics $x_{(1)}\le \cdots \le x_{(n)}$. Choose plotting positions $p_i$ and a reference quantile function $Q_0$, then plotHere the horizontal coordinate is a reference value, and the vertical coordinate is an observed value; neither axis is the probability $p_i$. The displayed plotting positions are our chosen convention, placing every level strictly between zero and one. They are not a claim that the $i$th order statistic equals that population quantile. For two statistical samples, use sample quantiles at matching levels, with a stated interpolation convention when their sizes differ. ^[ref]

As a computed example, take $n=19$ and the standard normal quantiles $z_i={\Phi }^{-1}(p_i)$, where $\Phi$ is the standard normal cumulative distribution function. Construct the ordered values $a_i=10+2z_i$. At ranks $1,10,19$, the plotting positions are $1/38,1/2,37/38$, the reference quantiles are approximately $-1.937932,0,1.937932$, and the constructed values are approximately $6.124137,10,13.875863$. Every point lies on $a=10+2z$ by construction. These 19 values are deliberately chosen, not random draws or real measurements.

The usefulness of a Q-Q plot is that it turns a comparison of probability distributions into geometry: if a reference random variable $Z$ is transformed into $X=\mu +\sigma Z$ with $\sigma >0$, its quantile function satisfies $Q_X(p)=\mu +\sigma Q_Z(p)$. This follows because a positive scaling preserves ordering. Thus the location-scale family produces a straight population Q-Q plot, and an approximate sample line suggests a compatible shape over the observed range. Its intercept describes location and its slope describes scale. The identity line requires the same location, scale and units on both axes; even then its displayed angle is 45 degrees only with equal axis scales. Curvature is often more informative than a histogram because it shows which quantiles depart from the reference. ^[ref]

A random affine recursion updates a real random variable by $X_n=A_n{X}_{n-1}+B_n$. Assume the coefficient pairs $(A_n,B_n)$ are independent and identically distributed random variables, independent of the initial value. The two coordinates within a pair may be dependent. Fresh coefficients make the recursion a time-homogeneous Markov process: its next conditional distribution depends on the current value alone.

The competition between the random multiplier and the fresh input determines whether initial conditions are forgotten. A synchronous coupling makes this particularly visible by using the same coefficient sequence for two initial values.

A random element is a measurable function from a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ to a measurable space $(E,\mathcal{E})$. Its law is the pushforward measure ${\mathbb{P}}_X(A)=\mathbb{P}(X^{-1}(A))$ for $A\in \mathcal{E}$. Real-valued random variables and vector-valued random variables are examples; a stochastic sample path can also be a random element when its path-space sigma-algebra is specified. For a Polish space $E$, the usual choice is its Borel sigma-algebra. A large deviation principle concerns the laws of a family of such random elements.

A random field is a family of random variables indexed by points of a spatial domain, or more generally by a multi-dimensional parameter set. Its componentwise covariance function and spectral tensor describe second-order dependence. A Gaussian random field has jointly Gaussian finite-dimensional distributions; a general random field need not be Gaussian.

A random matrix is a matrix whose entries are random variables. Its eigenvalues form a random configuration, and its empirical spectral measure records their distribution with equal mass on each eigenvalue. Entry assumptions and normalization determine the spectral regime.

A random measure assigns a measure ${\mu }_{\omega }$ to each outcome so that $\omega \mapsto {\mu }_{\omega }(A)$ is a random variable for every measurable $A$. A finite-variation process gives pathwise signed random measures through Lebesgue–Stieltjes integration.

A random sum has a random number $N$ of summands, usually with $N$ a nonnegative integer-valued random variable. The empty sum is zero. The dependence between $N$ and the summands must be stated; independent summands do not by themselves imply independence from $N$.

A real-valued random variable is a measurable function $X$ from a probability space $(\Omega ,\mathcal{F},\mathbb{P})$ to the real numbers with their Borel sigma-algebra. More general targets give random elements. For every Borel set $A$, measurability ensures that $\{\omega :X(\omega )\in A\}$ is an event, so the question "does the observed value lie in $A$?" has a defined probability.

The function $X$ is fixed; its input outcome is uncertain. This separates what we measure from how the uncertainty is modeled. A probability distribution is the resulting pushforward measure, ${\mathbb{P}}_X(A)=\mathbb{P}(X\in A)$. A cumulative distribution function, probability mass function, or probability density function describes that measure in an appropriate setting. The random variable and its probability distribution are different objects: the former also retains its relationship to other measurements on the same sample space.

For two independent uniform dice, let $U,V$ record their results and define $S=U+V$ and $M=\max (U,V)$. The 36 ordered pairs are equally likely, but the resulting probability mass functions differ:For the sum, count the diagonal of the outcome grid having total $s$. For the maximum, subtract the $(m-1{)}^2$ pairs with both results below $m$ from the $m^2$ pairs with both results at most $m$. These examples show how a deterministic function transforms a simple input probability distribution into a nonuniform output one.

The corresponding expected values are $\mathbb{E}S=7$ and $\mathbb{E}M=161/36$. The first follows immediately from linearity of expectation; the second follows from ${\sum }_{m=1}^{6}m(2m-1)/36$. Neither number must be an attainable result. An expected value describes an average under the probability measure, not a prediction that the next observation will equal it.

The distinction between a random variable and its probability distribution matters for dependence. $U$ and $V$ have the same discrete uniform distribution and are independent random variables. $U$ and a second copy of $U$ also have that same marginal probability distribution, but they always agree. Their joint probability distribution is concentrated on the diagonal; the marginals alone cannot say which relationship holds. This is why a model for several measurements must specify their joint law or an appropriate independence assumption.

Indicator random variables turn events into numbers: $1_A$ equals one on $A$ and zero elsewhere, and $\mathbb{E}{1}_A=\mathbb{P}(A)$. This connects event counting with expected value and variance, and lets tools for random variables answer questions about probabilities. More complicated functions of observations can describe totals, waiting times, or estimation errors using the same framework.

A random vector is a random variable whose values are vectors or, equivalently, an ordered tuple of scalar random variables on one probability space.

Take independent random variables equal to $1+n$ and $1-n$ with probability $1/(2n\log n)$ each, and otherwise equal to one. Their first absolute deviations are $1/\log n\to 0$, so $X_n\to 1$ in L1 convergence. The same triangle-inequality argument gives convergence in $L^1$ of their arithmetic averages to one. For $p>1$, however, $\mathbb{E}|X_n-1{|}^p=n^{p-1}/\log n\to \infty$.

The exceptional jumps occur infinitely often almost surely by the Borel-Cantelli second lemma, since $\sum 1/(n\log n)=\infty$. If the arithmetic averages $S_n/n$ converged to any finite value along a sample path, then $X_{n+1}/n=S_n/n-[(n-1)/n][S_{n-1}/(n-1)]$ would tend to zero. At each exceptional jump its magnitude is at least one. Thus the averages have no finite almost sure limit. Independence and convergence of first-moment averages do not alone give a strong law of large numbers.

The $r$th raw moment of a random variable $X$ is $\mathbb{E}[X^r]$, when it exists. A central moment instead uses $\mathbb{E}[(X-\mathbb{E}[X]{)}^r]$.

The Rayleigh distribution is the probability distribution of the Euclidean norm of a two-dimensional random vector with independent zero-mean normal random variables of equal variance as components. If $X,Y\sim N(0,{\sigma }^2)$ are independent, thenHere $\sigma$ is the standard deviation of each component, not of the radius $R$. At $\sigma =1$ this gives the unit-scale probability density function $r{e}^{-r^2/2}$.^[ref]

This geometry turns two signed errors into one nonnegative distance. For example, suppose horizontal and vertical positioning errors are independent $N(0,4{\mathrm{cm}}^2)$ random variables. Their means both vanish, but the distance from the intended point is usually positive. The Rayleigh distribution with $\sigma =2\mathrm{cm}$ gives the probability of landing inside any centered disk. These are chosen model assumptions, rather than measurements of a particular instrument.

An apparently paradoxical feature is that the most likely location per unit area is the origin, while the radial mode is $\sigma$. A narrow annulus farther out has more area. Near zero that growing circumference outweighs the falling planar density; farther out the exponential function wins. Distinguishing densities measured against different elements of area or length resolves the paradox.

For positive mixture weights $w_j$ summing to one, set $C_j={\sum }_{i\le j}{w}_i$ and choose $J=j$ when $C_{j-1}\le U<C_j$, where $U$ has a uniform distribution. Then $V=(U-C_{J-1})/w_J$ has a uniform distribution independently of $J$: for $0\le v\le 1$, $\mathrm{Pr}(J=j,V\le v)=w_{j}v$. This permits exact finite mixture model sampling with a recycled continuous random variable.

A regular conditional distribution of a random variable $X$ given $Y$ is a Markov kernel $K$ such that, for all measurable sets $A,B$,For each $y$, $K(y,\cdot )$ is a probability measure; for each $B$, $K(\cdot ,B)$ is measurable. Thus it assigns an actual probability distribution to a specified observation even when $P(Y=y)=0$. Such kernels exist for standard Borel state spaces, including real-valued variables, and their uniqueness is only up to $P_Y$-null sets. With a joint density and positive marginal density, the ratio of the two densities defines the usual conditional density kernel. Values outside the marginal support or on a null set are not uniquely determined by the joint law.

A reparameterization gradient differentiates an expected value by representing the simulated random variable as a differentiable transformation of parameter-independent noise. For $X_{\eta }=g_{\eta }(\epsilon )$, the identity ${\nabla }_{\eta }\mathbb{E}[F_{\eta }(X_{\eta })]=\mathbb{E}[{\nabla }_{\eta }{F}_{\eta }(g_{\eta }(\epsilon ))]$ requires justified differentiation under the integral sign. For a normal distribution, use $X=m+e^{\rho }\epsilon$ with $\epsilon$ drawn from the standard normal distribution.