Expected value ($\mathbb{E}[X]$)

An expected value is the probability-weighted arithmetic mean of a real-valued random variable. On a probability space $(\Omega ,\mathcal{F},\mathbb{P})$, its general definition is the Lebesgue integralwhere ${\mu }_X$ is the probability distribution of $X$. For an integrable random variable, this is a finite real number. With a probability mass function $p_X$, use ${\sum }_{x}x{p}_X(x)$; with a probability density function $f_X$, use ${\int }_{\mathbb{R}}x{f}_X(x)dx$. These are two ways of computing the same weighted average, rather than different definitions for different experiments.

For example, suppose $X$ takes the values $-1,2,8$ with probabilities $1/2,3/10,1/5$. ThenThe most likely outcome is $-1$, and $1.7$ is not a possible outcome at all. The expected value summarizes all outcomes and their weights; it need not predict the result of one trial. Geometrically, placing masses equal to the probability masses at the outcome positions gives a center of mass at $1.7$, since ${\sum }_x{p}_X(x)(x-1.7)=0$. This balance-point interpretation also explains the connection to a weighted arithmetic mean. ^[ref].

For the dimensionless random variable $Y$ with probability density function $f_Y(y)=2y$ on $[0,1]$, the expected value is $\mathbb{E}[Y]={\int }_0^{1}y(2y)dy=2/3$. Integrating the probability density function itself gives one; to compute the expected value, multiply by the outcome first. ^[ref].

Why is this useful? Linearity of expectation lets us compute an average total by adding average contributions, even for dependent random variables. And for independent and identically distributed integrable random variables, the strong law of large numbers says their sample mean approaches their common expected value almost surely. The theorem supplies the link to repeated observations; the weighted-average definition itself does not promise that any particular finite run will be close to the expected value.

There is a useful historical connection to valuing uncertain rewards. In his 1657 De ratiociniis in ludo aleae, Christiaan Huygens assigns the value $(a+b)/2$ to equally likely rewards $a,b$, and $(pa+qb)/(p+q)$ when they have $p,q$ equally likely chances. These are the finite weighted-average formulas for expected value. His opening discussion asks how to divide the stakes when a game stops before completion, connecting probability to a concrete question about an unfinished contest. ^[ref].

The expected value of a numerical characteristic under the target population's probability distribution, when it exists. A sample mean estimates this fixed statistical parameter from observed data; it is not itself the population value.

The law of the unconscious statistician computes the expected value of a measurable function of a random variable directly from the original probability distribution. If $g(X)$ is an integrable random variable, thenUse ${\sum }_{x}g(x)p_X(x)$ for a probability mass function, or ${\int }_{\mathbb{R}}g(x)f_X(x)dx$ for a probability density function. There is no need to derive the probability distribution of $g(X)$ first. The identity is the integration rule for a pushforward measure, and also holds with an infinite value when $g$ is nonnegative. ^[ref].

For $X$ taking $-1,2,8$ with probabilities $1/2,3/10,1/5$, choosing $g(x)=x^2$ gives $\mathbb{E}[X^2]=\frac{1}{2}+4\frac{3}{10}+64\frac{1}{5}=14.5$. Its variance is therefore $14.5-1.7^2=11.61$. Squaring the expected value would instead give $2.89$, because squaring does not commute with averaging.

For a bounded possibly nonmeasurable real function $Z$ on a probability space, its outer expectation is the infimum of expected values of measurable majorants. Its inner expectation is ${\mathbb{E}}_{\ast }Z=-{\mathbb{E}}^{\ast }(-Z)$. They agree with the usual expected value for measurable $Z$. This permits function-space convergence in distribution for an empirical distribution process whose paths are not Borel-measurable in a nonseparable norm topology: both expectations of each bounded continuous test functional must approach its ordinary expectation under the tight measurable limiting law.

For independent integrable variables, $\mathbb{E}{\prod }_i{X}_i={\prod }_i\mathbb{E}{X}_i$. Tonelli theorem first gives $\mathbb{E}{\prod }_i|X_i|={\prod }_i\mathbb{E}|X_i|<\infty$; then Fubini's theorem factors the signed joint integral.

A mean-zero function has expectation zero with respect to the specified probability measure. On a finite set with uniform measure this means ${\sum }_{x}f(x)=0$. A balanced subset indicator is an example. In Fourier analysis on a finite group, this is equivalent to vanishing of the component at the trivial representation.

A real or complex random variable $X$ is integrable when $\mathbb{E}|X|<\infty$. This is integrability with respect to its probability measure, and ensures that its expected value is finite. A bounded random variable is always an integrable random variable; so is the example taking $-1,2,8$, for which $\mathbb{E}|X|=2.7$.

For a real random variable, put $X_{+}=\max (X,0)$ and $X_{-}=\max (-X,0)$, its positive part and negative part. Each has a nonnegative Lebesgue integral, possibly infinite. The expression $\mathbb{E}[X]=\mathbb{E}[X_{+}]-\mathbb{E}[X_{-}]$ is defined when at least one of these two expected values is finite; if both are infinite, the expected value is undefined. Integrability makes both finite. A nonnegative random variable can have expected value $+\infty$, even though it is not an integrable random variable.

The standard Cauchy distribution is a useful warning. With its probability density function $f(x)=1/[\pi (1+x^2)]$, the positive contribution is ${\int }_0^{R}xf(x)dx=\log (1+R^2)/(2\pi )$, which tends to infinity; the negative contribution has the same magnitude. Integrating symmetrically over $[-R,R]$ gives zero for each $R$, but this cancellation does not define an expected value. The Cauchy sample mean has an unchanged distribution, showing why the integrability hypothesis in the strong law of large numbers matters.

For an integrable random variable, conditional expectation preserves integrability, and linearity of expectation can safely combine finite averages without forming $\infty -\infty$.

Let $Z$ have standard normal distribution, let $m\ge 1$ be an integer and let $\alpha \in \mathbb{R}$. A finite expected value of $Z{e}^{\alpha Z^{2m}}$ requires absolute integrability, which reduces to finiteness ofFor $m=1$ it is finite exactly when $\alpha <1/2$. For every $m\ge 2$ it is finite exactly when $\alpha \le 0$: the higher power dominates the negative quadratic when $\alpha >0$, and for $\alpha \le 0$ the integrand is bounded by $z{e}^{-z^2/2}$. In the finite cases the signed expected value is zero by symmetry. Otherwise each signed part has infinite integral; a formal symmetric cancellation is only a Cauchy principal value.

The $n$th raw moment of a random variable $X$ is $\mathbb{E}[X^n]$ when this expectation exists.

More generally, a moment is an integral weighted by a power of a coordinate against a measure or density; raw, central, and standardized moments are distinguished by the coordinate and normalization used.

The $r$th raw moment of a random variable $X$ is $\mathbb{E}[X^r]$, when it exists. A central moment instead uses $\mathbb{E}[(X-\mathbb{E}[X]{)}^r]$.

The fourth moment is $\mathbb{E}[X^4]$. Its finiteness is equivalent to square integrability of $X^2$. For centered variables, dividing it by the squared variance gives kurtosis.

A probability distribution is moment-determinate if its moments uniquely specify it among probability distributions. A moment-generating function finite in a neighbourhood of zero is a sufficient condition. The standard normal distribution is moment-determinate.

The second moment of a real random variable is $\mathbb{E}[X^2]$. Its variance is $\mathbb{E}[X^2]-(\mathbb{E}X{)}^2$ when these quantities are finite.

The absolute moment of order $p>0$ of a random variable $X$ is $\mathbb{E}[|X{|}^p]$ when this expectation is finite.

The $n$th central moment is $\mathbb{E}[(X-\mathbb{E}X{)}^n]$. The first central moment is zero and the second is the variance.

Skewness is the standardized third central moment of a probability distribution. A symmetric distribution with finite third moment has zero skewness. Cosmology also uses the reduced skewness $S_3=⟨{\delta }^3⟩/{\sigma }^4$, which differs from the standardized skewness by a factor of $\sigma$.

The variance measures the mean squared distance of a random variable from its expected value. For a real square-integrable random variable $X$,The square prevents positive and negative deviations from cancelling. It also gives the variance squared units: if $X$ measures a length in metres, its variance has units of square metres. Its nonnegative square root is the standard deviation, which has the same units as $X$.

For example, let $X$ take $-1$, $0$ and $1$ with equal probabilities. Its expected value is zero, but its variance is $[(-1{)}^2+0^2+1^2]/3=2/3$, and its standard deviation is $\sqrt{2/3}$. The constant random variable zero has the same expected value and zero variance: a mean alone does not describe the spread. The variance is zero exactly when $X=\mathbb{E}X$ almost surely.

The variance is useful because squared deviations interact neatly with linear combinations. The covariance supplies the cross terms when combining different random variables, and a covariance matrix organizes those terms for many variables. The characterization of variance as the minimum mean squared error of a constant also explains why the expected value is the best constant prediction under squared error.

For independent identically distributed random vectors $Q,Q^{\prime }$ with finite second absolute moments, expand $|Q-Q^{\prime }{|}^2=|Q{|}^2+|Q^{\prime }{|}^2-2Q\cdot Q^{\prime }$. Their independence makes $\mathbb{E}(Q\cdot Q^{\prime })=|\mathbb{E}Q{|}^2$, proving the identity. The right side is the sum of the coordinate variances. Applied to the displacement in an optimal transport plan, it separates motion of the barycenter of a measure from relative particle motion.

For a finite population $x_1,\dots ,x_n$ with arithmetic mean $\bar{x}$, its variance is $\frac{1}{n}{\sum }_{i=1}^n(x_i-\bar{x}{)}^2$. It averages squared deviations over the whole population.

For a nonnegative count with positive mean, the Fano factor compares its variance to its mean. A Poisson distribution has factor one. Correlated production bursts or weak restoring feedback can give factors larger than one even at large molecular copy number.

// Target: mathematical-biology.bigb

For a random variable with nonzero mean, the coefficient of variation is the dimensionless ratio

For a square-integrable real random variable $X$,Thus the unique best constant predictor is $a=\mathbb{E}X$, and the minimum mean squared error is $\mathrm{Var}(X)$.

The conditional variance measures the remaining squared uncertainty after specified information has been used to form a conditional mean. For a real square-integrable random variable $X$ and a sub-sigma-algebra $\mathcal{G}$, let $m=\mathbb{E}[X\mid \mathcal{G}]$. ThenIt is a $\mathcal{G}$-measurable random variable, nonnegative almost surely, rather than necessarily a single number. With the information $\mathcal{G}=\sigma (Y)$, its value is the variance in the group identified by $Y$. Its expected value is the mean squared error of predicting $X$ by its conditional mean.^[ref]

In the exact mixture distribution in law of total expectation, group $A$ contains $0,4$ around mean two, and group $B$ contains $6,10$ around mean eight. Both groups have conditional variance four because their deviations from their own means are $-2,2$ with equal probabilities. Therefore $\mathbb{E}[\mathrm{Var}(X\mid Y)]=4$. Meanwhile, the two group means have variance $\frac{1}{4}(2-13/2{)}^2+\frac{3}{4}(8-13/2{)}^2=27/4$. The law of total variance givesDirect calculation gives $\mathbb{E}[X^2]=53$ and $\mathrm{Var}(X)=53-(13/2{)}^2=43/4$, checking the decomposition.

For any square-integrable $\mathcal{G}$-measurable predictor $Z$, the error identity isThe cross term vanishes because $\mathbb{E}[X-m\mid \mathcal{G}]=0$. Thus using the conditional mean minimizes mean squared error, while conditional variance quantifies what this information still cannot predict. This is the orthogonal projection interpretation in L2 space.^[ref]

For square-integrable $X$ and nested information $\mathcal{H}\subseteq \mathcal{G}$, let $m_{\mathcal{H}}=\mathbb{E}[X\mid \mathcal{H}]$ and $m_{\mathcal{G}}=\mathbb{E}[X\mid \mathcal{G}]$. The predictor $m_{\mathcal{H}}$ is also $\mathcal{G}$-measurable, so the conditional variance error identity givesMore information cannot increase the average best squared prediction error. Equality holds exactly when the two conditional means agree almost surely; additional information can change the conditional distribution without changing its mean.

This is an averaged statement. Take $X=-4,0,4$ with equal probabilities, and learn only whether $X$ is zero. The unconditional variance is $32/3$. In the nonzero group, the conditional variance is $16$, which is larger; in the zero group it is zero. Their weighted average remains $(2/3)16+(1/3)0=32/3$. Both groups have conditional mean zero, so the extra information has not improved squared-error prediction, even though it has revealed whether the outcome is extreme or central. An individual group's conditional variance need not decrease.

For independent square-integrable random variables,

For independent random variables with ${\sup }_n\mathbb{E}|X_n{|}^2<\infty$, writeThen $\Vert Y_N-m_N{\Vert }_2^2=N^{-2}{\sum }_{k\le N}\mathrm{Var}(X_k)=O(N^{-1})$. Hence $Y_N$ converges in $L^2$ exactly when the scalar Cesàro means $m_N$ converge.

For square-integrable random variables,

Let $Z_1,\dots ,Z_n$ each be uniform on $[0,1]$, with arbitrary dependence. For centered unit-norm $f\in L^2[0,1]$, putThenPartition $[0,1]$ into $k$ equal intervals and test normalized differences of two interval indicators. Averaging over the two intervals reduces the claim to ${\sum }_p{N}_p^2\ge {\sum }_p{N}_p=n$ for their occupancy counts, and then let $k\to \infty$.

The standard deviation is the nonnegative square root ${\sigma }_X=\sqrt{\mathrm{Var}(X)}$ of the variance.

The covariance measures the average product of two centered deviations. For real square-integrable random variables $X$ and $Y$,Deviations with the same sign contribute positively; deviations with opposite signs contribute negatively. Their magnitudes and probabilities determine the net result. Taking $Y=X$ recovers the variance. The definitions and algebraic properties are presented in Orloff and Bloom, MIT 18.05, Covariance and Correlation, sections 2–2.1.

For a worked example, let $X,Z$ be independent random variables, each equally likely to be $-1$, $0$ or $1$, and put $Y=2X+Z$. All nine pairs $(X,Z)$ have probability $1/9$. The expected values are zero, while linearity of covariance and the vanishing covariance of independent random variables giveThe shared $X$ term makes large $X$ accompany large $Y$ on average. Replacing $2X$ by $-2X$ reverses the covariance to $-4/3$ without changing either variance.

The covariance is useful whenever deviations are added or subtracted. Expanding the square givesFor the worked model, these are $20/3$ and $4/3$. Positive covariance increases variation in the sum while reducing variation in the difference: the same shared fluctuation reinforces one combination and partly cancels in the other. The identity applies beyond independent random variables.

The size of a covariance depends on the measurement scale. For constants $a,b,c,d$, linearity of covariance gives $\mathrm{Cov}(aX+b,cY+d)=ac\mathrm{Cov}(X,Y)$. Adding offsets has no effect; rescaling either measurement rescales the covariance. The Pearson correlation coefficient divides by both standard deviations to describe the standardized linear relationship.

For mutually independent square-integrable random variables $A,B,C$, factor the joint moment as $\mathbb{E}[A{B}^{2}C]=\mathbb{E}A\mathbb{E}[B^2]\mathbb{E}C$, then subtract the product of the two means. This proves the formula. Sharing one factor can therefore produce nonzero covariance even when all original factors are independent. Products of disjoint groups of independent factors are themselves independent random variables.

For square-integrable random variables, expand $X$ and $Y$ into their conditional expectations and their centered conditional residuals. The cross terms vanish by the law of total expectation, giving the displayed covariance decomposition. Conditional independence can therefore coexist with positive unconditional covariance from a shared latent variable.

If $Z$ is nondegenerate, $g$ is strictly increasing and the relevant products are integrable, an independent copy $Z^{\prime }$ gives $2\mathrm{Cov}(g(Z),Z)=\mathbb{E}[(g(Z)-g(Z^{\prime }))(Z-Z^{\prime })]$. The integrand is positive exactly when $Z\ne Z^{\prime }$, so the covariance is strictly positive. The same argument applies within each conditional law and yields a positive terminal stock hedge for an attainable increasing payoff.

For square-integrable random variables, $\mathrm{Cov}(X,Z\mid \mathcal{G})=\mathbb{E}[XZ\mid \mathcal{G}]-\mathbb{E}[X\mid \mathcal{G}]\mathbb{E}[Z\mid \mathcal{G}]$. It is a $\mathcal{G}$-measurable random variable. Replacing $X$ by its conditional expectation given a larger sigma-field leaves this covariance unchanged when $Z$ is measurable with respect to that larger sigma-field. This observation underlies recovery of a martingale-transform integrand by conditional covariance.

For paired observations $(x_i,y_i)$, the sample correlation coefficient isIt is the normalized inner product of the centered data vectors, or the sample covariance divided by the product of the corresponding sample standard deviations. The choice of divisor $n$ or $n-1$ cancels when used consistently in all three quantities. It is undefined if either centered data vector is zero, which occurs when that variable is constant in the observed statistical sample.

For $x=(1,2,3,4)$ and $y=(2,1,4,3)$, both sample means are $2.5$. The sum of centered products is $3$, and each sum of centered squares is $5$, hence the sample correlation coefficient is $3/5=0.6$. This is a summary of these four observed pairs; it is not automatically the population Pearson correlation coefficient. Philip Stark, Computing the correlation coefficient explains the centered and standardized calculation.

A sample correlation matrix rescales a sample covariance matrix $S$ to unit diagonal, for variables with positive sample variance. Its off-diagonal entries are sample correlation coefficients. Strong correlations between predictors can indicate multicollinearity, but pairwise correlations alone do not exclude higher-order linear dependence.

For square-integrable complex random variables, the Hermitian covariance is ${\mathrm{Cov}}_{\mathbb{C}}(Z,W)=\mathbb{E}[(Z-\mathbb{E}Z)\overline{(W-\mathbb{E}W)}]$. Its diagonal is nonnegative and equals $\mathbb{E}|Z-\mathbb{E}Z{|}^2$. The conjugation distinguishes it from the pseudo-covariance.

For a centered complex random variable $Z=A+iB$, the pseudo-covariance is $P=\mathbb{E}{Z}^2$. Together with $V=\mathbb{E}|Z{|}^2$ it determines the real covariance matrix: $\mathbb{E}{A}^2=(V+\mathrm{Re}P)/2$, $\mathbb{E}{B}^2=(V-\mathrm{Re}P)/2$, and $\mathbb{E}AB=\mathrm{Im}P/2$.

Covariance is bilinear: for constants $a_i,b_j$ and square-integrable random variables,

Two square-integrable random variables $X,Y$ are uncorrelated random variables when $\mathrm{Cov}(X,Y)=0$. Independent random variables with finite second moments are uncorrelated random variables, but zero covariance does not generally imply independence.

For an explicit counterexample, let $X$ take $-1$, $0$, $1$ with equal probabilities, and let $Y=X^2$. Then $\mathbb{E}X=\mathbb{E}{X}^3=0$, so $\mathrm{Cov}(X,Y)=\mathbb{E}{X}^3-\mathbb{E}X\mathbb{E}{X}^2=0$. Nevertheless, $Y$ is completely determined by $X$: $\mathbb{P}(Y=0\mid X=0)=1$, while $\mathbb{P}(Y=0)=1/3$. Thus these are uncorrelated random variables with statistical dependence. In this example, opposite contributions from the odd function $x^3$ cancel; the even function $x^2$ still retains information about $|X|$. The last panel of the covariance diagram shows the relationship.

There are useful special cases. Uncorrelated jointly normal variables are independent, and the Bernoulli independence from zero covariance result supplies another setting where the converse holds. Their distribution assumptions do the extra work; zero covariance alone does not.

Here is an explicit joint distribution, for $0<p<0.8$: assign mass $(1+p)e^{-p}-p$ at $(0,0)$, mass $p(1-e^{-p})$ at $(1,0)$, mass $p{e}^{-p}$ at $(1,1)$, and mass $e^{-p}{p}^k/k!$ at $(0,k)$ for $k\ge 2$; all other masses are zero. These are nonnegative: the first is decreasing in $p$, and at $p=0.8$ it is still positive. Summing over rows and columns gives a Bernoulli distribution with mean $p$ and a Poisson distribution with mean $p$, so the joint mass sums to one.

Only $(1,1)$ contributes to the mixed first moment, giving $\mathbb{E}[XY]=p{e}^{-p}$ and the covariance above. The strictly decreasing function $e^{-p}-p$ has a unique zero in $(0,0.8)$, approximately $0.567143$. At that parameter the variables are uncorrelated. They are nevertheless dependent, because $\mathbb{P}(X=1\mid Y=1)=1$ whereas $\mathbb{P}(X=1)=p<1$. The Bernoulli independence from zero covariance result requires both variables to be Bernoulli and does not apply here.

For independent standard normal random variables ${\eta }_1,\dots ,{\eta }_n$, define the displayed errors for $i\ge 2$. Each has mean zero and variance one. Nonadjacent errors depend on disjoint sets of independent variables. Adjacent covariances vanish because their product contains an independent centered factor. Thus their covariance matrix is $I_n$.

They are nevertheless dependent: the Gaussian fourth moment gives $\mathbb{E}[{ϵ}_i^2{ϵ}_{i+1}^2]=3$, and hence $\mathrm{Cov}({ϵ}_i^2,{ϵ}_{i+1}^2)=2$. Constant variance and zero pairwise covariance can conceal dependence in magnitudes. This is useful when interpreting a residual plot in linear regression.

For two Bernoulli random variables with joint probabilities $a,b,c,d$ in the order $(0,0),(0,1),(1,0),(1,1)$, their covariance is $d-(c+d)(b+d)=ad-bc$. Zero covariance fixes the $(1,1)$ probability to the product of its marginals; subtracting this from each marginal fixes all three remaining joint probabilities to their products. Thus uncorrelated random variables taking only zero and one are independent, including degenerate marginals. The implication is special to this two-point setting and fails for general distributions.

For a real random vector $X=(X_1,\dots ,X_p{)}^T$ with finite second moments, its covariance matrix collects all pairwise covariances:The diagonal entries are variances. The covariance matrix is a symmetric matrix and a positive semidefinite matrix, because for every real vector $a$,Thus one matrix describes the variance of every linear combination of the coordinates. It is a positive-definite matrix exactly when no nonzero linear combination is constant almost surely. A deterministic linear relation produces a zero eigenvalue.

For example,has unit coordinate variances, covariance $0.8$, and eigenvalues $1.8$ and $0.2$. Its unit eigenvectors $(1,1{)}^T/\sqrt{2}$ and $(1,-1{)}^T/\sqrt{2}$ describe directions of greatest and least projected variance. For a unit vector $a=(\cos \theta ,\sin \theta {)}^T$, direct multiplication gives $a^T\Sigma a=1+0.8\sin (2\theta )$: the variation in the sum direction is nine times that in the difference direction.

The spectral theorem supplies an orthonormal basis of these directions. This connects covariance matrices to principal component analysis: its leading eigenvector identifies the unit-direction projection with the largest variance. For a linear transformation $A$, the transformed random vector has covariance matrix $A\Sigma A^T$, so uncertainties propagate through a linear model by matrix multiplication.

For a two-dimensional positive-definite covariance matrix $\Sigma$, the centered covariance ellipse is the image of the unit circle under its positive definite square root of a matrix. Equivalently, it satisfies $x^T{\Sigma }^{-1}x=1$. Its semi-major and semi-minor axes have lengths given by the square roots of the larger and smaller eigenvalues, along the corresponding eigenvectors. The ellipse encodes a second-moment geometry; a probability-coverage interpretation needs additional information about the probability distribution and the chosen scale.

For a random vector with covariance $\Sigma$ and positive coordinate variances, let $D=\mathrm{diag}(\Sigma )$. Its correlation matrix is the covariance of $D^{-1/2}(X-\mathbb{E}X)$, so it is positive semidefinite with unit diagonal. Each entry is the correlation of the corresponding pair of variables. In particular $\mathrm{tr}(C)=p$, explaining why an eigenvalue in correlation-based principal component analysis represents that eigenvalue divided by $p$ of the total variance. The empirical version is the sample correlation matrix.

This unit-variance covariance matrix has eigenvalue $1+(p-1)r$ on the constant-vector direction and eigenvalue $1-r$ on its orthogonal complement. It is positive semidefinite exactly when $-1/(p-1)\le r\le 1$. For positive $r$, the first population principal component is the normalized sum of the centered variables.

For a positive-definite covariance matrix, this is Euclidean distance after whitening. It accounts for both scales and correlations and is invariant under consistent nonsingular linear changes of coordinates. Estimating the covariance can be unstable with small samples, and a singular covariance requires restricting to a nonsingular subspace rather than claiming an ordinary metric on the full space.

A compound-symmetry covariance matrix has a common diagonal entry and a common off-diagonal entry: $\Sigma =aI+cJ$. For a group of size $k$, its within-group eigenvalue is $a$ and its constant-direction eigenvalue is $a+kc$. Thus $a\ge 0$ and $a+kc\ge 0$ characterize positive semidefiniteness. When $c\ge 0$, an independent shared random intercept plus individual noise realizes this covariance. A group mean has variance $c+a/k$; a difference of two disjoint subgroup means within the same group cancels $c$.

For a unit vector $u$ and $\theta >0$, a rank-one covariance spike is the covariance matrix $I+\theta u{u}^{\top }$. Its eigenvalue is $1+\theta$ in the direction $u$ and one on the orthogonal complement. It arises by adding an independent scalar normal distribution signal $Yu$ to a standard Gaussian random vector.

The sample mean and covariance estimate the first two moments of a random vector from repeated observations.

The uncentered empirical second-moment matrix averages the outer products of the observations. Its expected value is $\mathrm{Cov}(X)+\mathbb{E}[X]\mathbb{E}[X{]}^{\top }$. It is an unbiased estimator of the covariance matrix when the population expected value is known to be zero. It differs from a sample covariance matrix centered at the sample mean.

For observations $X_1,\dots ,X_n$, the sample mean is $\overline{X}=n^{-1}{\sum }_{i=1}^n{X}_i$.

For independent samples of sizes $n,m$ from populations with means ${\mu }_X,{\mu }_Y$ and finite variances ${\sigma }_X^2,{\sigma }_Y^2$, the difference of their sample means has expected value ${\mu }_X-{\mu }_Y$ and variance ${\sigma }_X^2/n+{\sigma }_Y^2/m$. This uses independence between and within the samples. With dependent samples, a covariance term is required instead.

Partition a sample into two disjoint groups $A,B$ with fixed positive sizes $n_A,n_B$, and let $w=n_A/(n_A+n_B)$. The overall sample mean is the weighted meanSubtracting gives the displayed identity. The subgroup and overall sample means share observations, so they are not independent. If the two disjoint groups are sampled independently, thenThus treating the subgroup and the combined sample as two independent samples uses the wrong standard error. Equality of the subgroup and overall population means is equivalent to equality of subgroup and complement means when both groups have positive weight, but the sampling distribution must respect the shared observations.

For independent and identically distributed observations with finite positive variance, the central limit theorem, consistency of the sample variance and Slutsky's theorem give $T_n\stackrel{d}{\to }N(0,1)$. Under an independent normal distribution sample it instead has the exact finite-sample Student t-distribution with $n-1$ degrees of freedom, as described by the studentized normal sample mean. The asymptotic statement does not require normal individual observations.

For observations $x_1,\dots ,x_n$, the sample covariance matrix is $n^{-1}{\sum }_i(x_i-\bar{x})(x_i-\bar{x}{)}^T$, or the same sum divided by $n-1$ under the unbiased convention.

If $H=I_n-11^T/n$ and $S=X^{T}HX/n$, then $\mathrm{rank}S=\mathrm{rank}(HX)\le \min (p,n-1)$. In particular full column rank of raw $X$ does not ensure a nonsingular centered sample covariance matrix. When $S$ is singular, its Gaussian precision profile likelihood has no finite maximizer: increase precision along a nonzero null vector.

For paired scalar observations $(x_i,y_i)$, their sample covariance is $n^{-1}{\sum }_i(x_i-\bar{x})(y_i-\bar{y})$, or the same sum divided by $n-1$ under the unbiased convention. It is an off-diagonal entry of the sample covariance matrix of the paired vectors.

The effective rank measures the total variance of a covariance matrix relative to its largest directional variance.

For independent centered Gaussian vectors with covariance $\Sigma$ and uncentered sample covariance $\hat{\Sigma }=n^{-1}{\sum }_i{x}_i{x}_i^T$, with probability at least $1-e^{-t}$,

The precision matrix is the inverse of a nonsingular covariance matrix. For a multivariate normal distribution, a zero off-diagonal precision entry encodes conditional independence of the corresponding coordinates given all remaining coordinates.

For a nonsingular multivariate normal distribution with precision matrix $\Omega$, fixing all coordinates except $X_i$ leaves a density proportional to $\exp [-{\Omega }_{ii}(x_i-m_i{)}^2/2]$ after completing the square. Thus its conditional variance is the reciprocal diagonal precision entry. This is conditioning on every other coordinate, not the marginal variance $({\Omega }^{-1}{)}_{ii}$.

In a Gaussian graphical model, regressing component $k$ on the others gives the displayed coefficients, residual variance $1/{\Omega }_{kk}$ and intercept ${\mu }_k-{\mu }_{-k}^{\mathsf{T}}{\theta }_{-k,k}$. Opposite directions have identical zero patterns and signs because $\Omega$ is symmetric with positive diagonal, but generally unequal coefficient magnitudes.

For independent multivariate normal observations with unknown mean and nonsingular covariance, profiling the mean at its sample value gives negative log-likelihood proportional to $-\log \det \Omega +\mathrm{tr}(S\Omega )$ over $\Omega \succ 0$. The sample covariance matrix uses divisor $n$. Its minimizer exists exactly when $S\succ 0$, and is then $S^{-1}$.

If a positive-definite matrix $Q=L{L}^T$ is the precision matrix of a multivariate normal distribution, draw independent $\xi \sim N(0,I)$ and solve $L^T\eta =\xi$. Then $m+\eta$ has covariance $L^{-T}{L}^{-1}=Q^{-1}$. This avoids forming the inverse and exploits a banded Cholesky decomposition when $Q$ is banded.

The constrained ${\ell }^1$ minimization estimator minimizes $\Vert \Omega {\Vert }_1$ subject toIts constraints and objective separate by columns. Feasibility of the true precision matrix and elementary matrix-norm inequalities yield entrywise error bounds under an entrywise covariance-estimation bound.

The graphical Lasso estimates a sparse precision matrix by minimizing a Gaussian negative log-likelihood plus an entrywise ${\ell }^1$ penalty.

For a Graphical Lasso solution with inverse $\hat{\Sigma }$, the unique minimizer of $\frac{1}{2}\Vert Wb-W^{-1}{S}_{-j,j}{\Vert }_2^2+\lambda \Vert b{\Vert }_1$, $W^2={\hat{\Sigma }}_{-j,-j}\succ 0$, is $b=-{\hat{\Omega }}_{-j,j}/{\hat{\Omega }}_{jj}$. The inverse identity gives ${\hat{\Sigma }}_{-j,j}={\hat{\Sigma }}_{-j,-j}b$, and the off-diagonal Graphical-Lasso Karush-Kuhn-Tucker conditions become the column Lasso conditions because the corresponding signs are opposite.

For objective $-\log \det \Omega +\mathrm{Tr}(S\Omega )+\lambda \Vert \Omega {\Vert }_{1,\mathrm{entry}}$, the KKT conditions are $-{\Omega }^{-1}+S+\lambda Z=0$ with $Z_{ij}\in \partial |{\Omega }_{ij}|$.

The Pearson correlation coefficient is the covariance divided by the product of the standard deviations. For real random variables with finite, strictly positive variances,It is the expected value of the product of the two centered variables after dividing each by its standard deviation. The Cauchy-Schwarz inequality gives $-1\le {\rho }_{X,Y}\le 1$. This removes measurement units, while retaining the direction and strength of the linear relationship. Constant random variables have zero standard deviation, so their Pearson correlation coefficient is undefined.