Expected value ($\mathbb{E}[X]$)

The expected value is the probability-weighted average of a random variable, defined by a sum or an integral when it exists.

For independent integrable variables, $\mathbb{E}{\prod }_i{X}_i={\prod }_i\mathbb{E}{X}_i$. Tonelli theorem first gives $\mathbb{E}{\prod }_i|X_i|={\prod }_i\mathbb{E}|X_i|<\infty$; then Fubini's theorem factors the signed joint integral.

A mean-zero function has expectation zero with respect to the specified probability measure. On a finite set with uniform measure this means ${\sum }_{x}f(x)=0$. A balanced subset indicator is an example. In Fourier analysis on a finite group, this is equivalent to vanishing of the component at the trivial representation.

A real or complex random variable $X$ is integrable when $\mathbb{E}|X|<\infty$. Its expected value is then finite, and conditional expectation preserves integrability.

The $n$th raw moment of a random variable $X$ is $\mathbb{E}[X^n]$ when this expectation exists.

The fourth moment is $\mathbb{E}[X^4]$. Its finiteness is equivalent to square integrability of $X^2$. For centered variables, dividing it by the squared variance gives kurtosis.

A probability distribution is moment-determinate if its moments uniquely specify it among probability distributions. A moment-generating function finite in a neighbourhood of zero is a sufficient condition. The standard normal distribution is moment-determinate.

The second moment of a real random variable is $\mathbb{E}[X^2]$. Its variance is $\mathbb{E}[X^2]-(\mathbb{E}X{)}^2$ when these quantities are finite.

The absolute moment of order $p>0$ of a random variable $X$ is $\mathbb{E}[|X{|}^p]$ when this expectation is finite.

The $n$th central moment is $\mathbb{E}[(X-\mathbb{E}X{)}^n]$. The first central moment is zero and the second is the variance.

Skewness is the standardized third central moment of a probability distribution. A symmetric distribution with finite third moment has zero skewness. Cosmology also uses the reduced skewness $S_3=⟨{\delta }^3⟩/{\sigma }^4$, which differs from the standardized skewness by a factor of $\sigma$.

The variance of a square-integrable random variable is

For a random variable with nonzero mean, the coefficient of variation is the dimensionless ratio

For a square-integrable real random variable $X$,Thus the unique best constant predictor is $a=\mathbb{E}X$, and the minimum mean squared error is $\mathrm{Var}(X)$.

The conditional variance isIts expectation is the mean squared error of the conditional-expectation predictor.

For independent square-integrable random variables,

For independent random variables with ${\sup }_n\mathbb{E}|X_n{|}^2<\infty$, writeThen $\Vert Y_N-m_N{\Vert }_2^2=N^{-2}{\sum }_{k\le N}\mathrm{Var}(X_k)=O(N^{-1})$. Hence $Y_N$ converges in $L^2$ exactly when the scalar Cesaro means $m_N$ converge.

For square-integrable random variables,

Let $Z_1,\dots ,Z_n$ each be uniform on $[0,1]$, with arbitrary dependence. For centered unit-norm $f\in L^2[0,1]$, putThenPartition $[0,1]$ into $k$ equal intervals and test normalized differences of two interval indicators. Averaging over the two intervals reduces the claim to ${\sum }_p{N}_p^2\ge {\sum }_p{N}_p=n$ for their occupancy counts, and then let $k\to \infty$.

The standard deviation is the nonnegative square root ${\sigma }_X=\sqrt{\mathrm{Var}(X)}$ of the variance.

The covariance of square-integrable random variables isIt determines the variance of a difference through

For square-integrable random variables, expand $X$ and $Y$ into their conditional expectations and their centered conditional residuals. The cross terms vanish by the law of total expectation, giving the displayed covariance decomposition. Conditional independence can therefore coexist with positive unconditional covariance from a shared latent variable.

If $Z$ is nondegenerate, $g$ is strictly increasing and the relevant products are integrable, an independent copy $Z^{\prime }$ gives $2\mathrm{Cov}(g(Z),Z)=\mathbb{E}[(g(Z)-g(Z^{\prime }))(Z-Z^{\prime })]$. The integrand is positive exactly when $Z\ne Z^{\prime }$, so the covariance is strictly positive. The same argument applies within each conditional law and yields a positive terminal stock hedge for an attainable increasing payoff.

For square-integrable random variables, $\mathrm{Cov}(X,Z\mid \mathcal{G})=\mathbb{E}[XZ\mid \mathcal{G}]-\mathbb{E}[X\mid \mathcal{G}]\mathbb{E}[Z\mid \mathcal{G}]$. It is a $\mathcal{G}$-measurable random variable. Replacing $X$ by its conditional expectation given a larger sigma-field leaves this covariance unchanged when $Z$ is measurable with respect to that larger sigma-field. This observation underlies recovery of a martingale-transform integrand by conditional covariance.

The sample correlation coefficient is the centered sample covariance divided by the product of sample standard deviations. It is the normalized inner product of centered data vectors and is undefined when either vector is constant.

For square-integrable complex random variables, the Hermitian covariance is ${\mathrm{Cov}}_{\mathbb{C}}(Z,W)=\mathbb{E}[(Z-\mathbb{E}Z)\overline{(W-\mathbb{E}W)}]$. Its diagonal is nonnegative and equals $\mathbb{E}|Z-\mathbb{E}Z{|}^2$. The conjugation distinguishes it from the pseudo-covariance.

For a centered complex random variable $Z=A+iB$, the pseudo-covariance is $P=\mathbb{E}{Z}^2$. Together with $V=\mathbb{E}|Z{|}^2$ it determines the real covariance matrix: $\mathbb{E}{A}^2=(V+\mathrm{Re}P)/2$, $\mathbb{E}{B}^2=(V-\mathrm{Re}P)/2$, and $\mathbb{E}AB=\mathrm{Im}P/2$.

Covariance is bilinear: for constants $a_i,b_j$ and square-integrable random variables,

Two square-integrable random variables $X,Y$ are uncorrelated when $\mathrm{Cov}(X,Y)=0$. Independent random variables with finite second moments are uncorrelated, but the converse generally fails.

For a random vector $X$ with finite second moments, its covariance matrix isIt is positive semidefinite, and the variance of $a^{T}X$ is $a^T\mathrm{cov}(X)a$.

A compound-symmetry covariance matrix has a common diagonal entry and a common off-diagonal entry: $\Sigma =aI+cJ$. For a group of size $k$, its within-group eigenvalue is $a$ and its constant-direction eigenvalue is $a+kc$. Thus $a\ge 0$ and $a+kc\ge 0$ characterize positive semidefiniteness. When $c\ge 0$, an independent shared random intercept plus individual noise realizes this covariance. A group mean has variance $c+a/k$; a difference of two disjoint subgroup means within the same group cancels $c$.

For a unit vector $u$ and $\theta >0$, a rank-one covariance spike is the covariance matrix $I+\theta u{u}^{\top }$. Its eigenvalue is $1+\theta$ in the direction $u$ and one on the orthogonal complement. It arises by adding an independent scalar normal distribution signal $Yu$ to a standard Gaussian random vector.

The sample mean and covariance estimate the first two moments of a random vector from repeated observations.

The uncentered empirical second-moment matrix averages the outer products of the observations. Its expected value is $\mathrm{Cov}(X)+\mathbb{E}[X]\mathbb{E}[X{]}^{\top }$. It is an unbiased estimator of the covariance matrix when the population expected value is known to be zero. It differs from a sample covariance matrix centered at the sample mean.

For observations $X_1,\dots ,X_n$, the sample mean is $\overline{X}=n^{-1}{\sum }_{i=1}^n{X}_i$.

For observations $x_1,\dots ,x_n$, the sample covariance matrix is $n^{-1}{\sum }_i(x_i-\bar{x})(x_i-\bar{x}{)}^T$, or the same sum divided by $n-1$ under the unbiased convention.

If $H=I_n-11^T/n$ and $S=X^{T}HX/n$, then $\mathrm{rank}S=\mathrm{rank}(HX)\le \min (p,n-1)$. In particular full column rank of raw $X$ does not ensure a nonsingular centered sample covariance matrix. When $S$ is singular, its Gaussian precision profile likelihood has no finite maximizer: increase precision along a nonzero null vector.

For paired scalar observations $(x_i,y_i)$, their sample covariance is $n^{-1}{\sum }_i(x_i-\bar{x})(y_i-\bar{y})$, or the same sum divided by $n-1$ under the unbiased convention. It is an off-diagonal entry of the sample covariance matrix of the paired vectors.

The effective rank measures the total variance of a covariance matrix relative to its largest directional variance.

For independent centered Gaussian vectors with covariance $\Sigma$ and uncentered sample covariance $\hat{\Sigma }=n^{-1}{\sum }_i{x}_i{x}_i^T$, with probability at least $1-e^{-t}$,

The precision matrix is the inverse of a nonsingular covariance matrix. For a multivariate normal distribution, a zero off-diagonal precision entry encodes conditional independence of the corresponding coordinates given all remaining coordinates.

In a Gaussian graphical model, regressing component $k$ on the others gives the displayed coefficients, residual variance $1/{\Omega }_{kk}$ and intercept ${\mu }_k-{\mu }_{-k}^{\mathsf{T}}{\theta }_{-k,k}$. Opposite directions have identical zero patterns and signs because $\Omega$ is symmetric with positive diagonal, but generally unequal coefficient magnitudes.

For independent multivariate normal observations with unknown mean and nonsingular covariance, profiling the mean at its sample value gives negative log-likelihood proportional to $-\log \det \Omega +\mathrm{tr}(S\Omega )$ over $\Omega \succ 0$. The sample covariance matrix uses divisor $n$. Its minimizer exists exactly when $S\succ 0$, and is then $S^{-1}$.

If a positive-definite matrix $Q=L{L}^T$ is the precision matrix of a multivariate normal distribution, draw independent $\xi \sim N(0,I)$ and solve $L^T\eta =\xi$. Then $m+\eta$ has covariance $L^{-T}{L}^{-1}=Q^{-1}$. This avoids forming the inverse and exploits a banded Cholesky decomposition when $Q$ is banded.

The constrained ${\ell }^1$ minimization estimator minimizes $\Vert \Omega {\Vert }_1$ subject toIts constraints and objective separate by columns. Feasibility of the true precision matrix and elementary matrix-norm inequalities yield entrywise error bounds under an entrywise covariance-estimation bound.

The graphical Lasso estimates a sparse precision matrix by minimizing a Gaussian negative log-likelihood plus an entrywise ${\ell }^1$ penalty.

For a Graphical Lasso solution with inverse $\hat{\Sigma }$, the unique minimizer of $\frac{1}{2}\Vert Wb-W^{-1}{S}_{-j,j}{\Vert }_2^2+\lambda \Vert b{\Vert }_1$, $W^2={\hat{\Sigma }}_{-j,-j}\succ 0$, is $b=-{\hat{\Omega }}_{-j,j}/{\hat{\Omega }}_{jj}$. The inverse identity gives ${\hat{\Sigma }}_{-j,j}={\hat{\Sigma }}_{-j,-j}b$, and the off-diagonal Graphical-Lasso Karush-Kuhn-Tucker conditions become the column Lasso conditions because the corresponding signs are opposite.

For objective $-\log \det \Omega +\mathrm{Tr}(S\Omega )+\lambda \Vert \Omega {\Vert }_{1,\mathrm{entry}}$, the KKT conditions are $-{\Omega }^{-1}+S+\lambda Z=0$ with $Z_{ij}\in \partial |{\Omega }_{ij}|$.

For random variables with finite positive variances,

For square-integrable variables of positive variance, correlation one is equivalent to $X-\mathbb{E}X=c(Y-\mathbb{E}Y)$ with $c>0$. The squared norm of the difference between the two standardized centered variables is $2(1-\rho )$, proving the equivalence. For uncentered variables the relationship can have a nonzero intercept; it need not be proportionality.

For two strictly positive zero-rate stock martingales with bounded volatilities and the same initial value, proportional terminal prices must be equal because their expectations coincide. Conditional expectation then makes the whole paths equal. Comparing their stochastic integrals by the Itô isometry gives equality of the volatility coefficients almost everywhere in time and probability. Perfect positive correlation alone does not provide the required proportionality.

The stocks $S_t=e^{B_t-t/2}$ and $S_t^{\prime }=(1+S_t)/2$ start at one, are true square-integrable zero-rate martingales, and have perfect positive correlation at each positive time. Their bounded multiplicative volatilities are $1$ and $S_t/(1+S_t)$, which differ throughout. The additive intercept is what allows the same correlation but different stochastic-exponential coefficients.

Responses observed on the same process or individual at different times may have nonzero correlation coefficients. Such serial correlation can arise from a shared random effect or from history-dependent conditional means. A scalar dispersion parameter adjustment alone does not represent this temporal dependence.

The intraclass correlation coefficient measures the correlation between observations in the same group. In a random-intercept model with group variance ${\sigma }_G^2$ and individual variance ${\sigma }_I^2$, it is ${\sigma }_G^2/({\sigma }_G^2+{\sigma }_I^2)$.

If a random variable $X$ takes values in $[m,M]$, then

For nonnegative $Y$, $E[Y]={\int }_0^{\infty }P(Y\ge t)dt$; for nonnegative integer $N$, $E[N]={\sum }_{k\ge 1}P(N\ge k)$.

For a nonnegative integer-valued variable, $X={\sum }_{m\ge 0}{1}_{\{X>m\}}$ and the tail-sum formula gives $\mathbb{E}X={\sum }_{m\ge 0}\mathbb{P}(X>m)$. For arbitrary positive real values, that integer-indexed sum equals $\mathbb{E}\lceil X\rceil$ instead. Two independent constants equal to $3/2$ demonstrate that the product of such tail sums need not equal the tail sum of their product: the two sides are four and three.

Expectation is linear without requiring independence: $E[aX+bY]=aE[X]+bE[Y]$ whenever the expectations exist.

For independent identically distributed variables of variance ${\sigma }^2$, the mean of $n$ samples has variance ${\sigma }^2/n$.

For $q>0$ and a random variable $X$ with finite absolute $q$th moment, Tonelli theorem gives

If $\mathbb{P}(|Z|\ge t)\le A{e}^{-b{t}^2}$ for $A\ge 1$ and $b>0$, then $\mathbb{E}{Z}^2\le (1+\log A)/b$. Indeed the tail integral formula for moments gives $\mathbb{E}{Z}^2\le {\int }_0^{\infty }\min (1,A{e}^{-bu})du$; split at $u=(\log A)/b$. Clipping at one improves the constant compared with integrating the exponential bound alone.