= Sample mean and covariance
{title2=$\overline X,S$}
{wiki}
For observations $X_1,\ldots,X_n\in\mathbb R^p$, the <sample mean> and empirical <covariance matrix> describe average location and joint variation:
$$
\overline X=\frac1n\sum_iX_i,\qquad S_n=\frac1n\sum_i(X_i-\overline X)(X_i-\overline X)^T.
$$
Each summand is an <outer product>, so $S_n$ is <positive semidefinite>. If the observations are <independent and identically distributed random variables> with <covariance matrix> $\Sigma$, then $\mathbb ES_n=(n-1)\Sigma/n$. For $n\geq2$, replacing the divisor $n$ by $n-1$ gives an <unbiased estimator>.
For the three observations $(0,0),(1,2),(2,1)$, the <sample mean> is $(1,1)$. The sum of centered <outer products> is
$$
\begin{pmatrix}2&1\\1&2\end{pmatrix},
$$
so the $n$-divisor <sample covariance matrix> is $\begin{pmatrix}2/3&1/3\\1/3&2/3\end{pmatrix}$, and the $n-1$ version is $\begin{pmatrix}1&1/2\\1/2&1\end{pmatrix}$. Stating the divisor avoids ambiguity when comparing formulas. The <rank of a centered sample covariance matrix> also explains why too few observations force singularity in high dimension.
Back to article page
