= Cross-covariance matrix
{title2=$C_{ab}=\operatorname{Cov}(\mathbf a,\mathbf b)$}
For real <random vectors> $\mathbf a\in\mathbb R^p$ and $\mathbf b\in\mathbb R^q$ with finite <second moments>, their <cross-covariance matrix> collects the <covariances> between the two sets of coordinates:
$$
C_{ab}=\mathbb E[(\mathbf a-\mathbb E\mathbf a)(\mathbf b-\mathbb E\mathbf b)^T],\qquad
(C_{ab})_{ij}=\operatorname{Cov}(a_i,b_j).
$$
It has size $p\times q$, and exchanging the two vectors transposes it: $C_{ba}=C_{ab}^T$. Unlike a <covariance matrix> of a vector with itself, a <cross-covariance matrix> need not be square or <symmetric>.
For an original construction, let centered independent <random variables> $X,Z$ have <variances>1and4, and put
$$
\mathbf a=\begin{pmatrix}X\\2X+Z\end{pmatrix},\qquad b=3X-Z.
$$
<Linearity of covariance> gives
$$
C_{ab}=\begin{pmatrix}3\\6-4\end{pmatrix}=\begin{pmatrix}3\\2\end{pmatrix}.
$$
The second entry contains a negative contribution from $Z$, although the total is positive. Also $\operatorname{Var}(b)=13$ and
$$
\Sigma_a=\begin{pmatrix}1&2\\2&8\end{pmatrix},\qquad
\operatorname{Cov}\begin{pmatrix}\mathbf a\\b\end{pmatrix}
=\begin{pmatrix}\Sigma_a&C_{ab}\\C_{ab}^T&13\end{pmatrix}.
$$
This joint <covariance matrix> is <positive semidefinite>. That restriction couples its diagonal and off-diagonal blocks; an arbitrary rectangular <matrix> cannot be combined with arbitrary within-vector <covariances> to produce a valid joint model.
For fixed real <matrices> $A,B$ and constant vectors $c,d$, direct substitution into the definition gives
$$
\operatorname{Cov}(A\mathbf a+c,B\mathbf b+d)=A C_{ab}B^T.
$$
This identity propagates shared input fluctuations through two different linear readouts, for example when assessing <shared calibration error>.
From $n$ independent identically distributed paired observations, the usual unbiased estimate is
$$
\widehat C_{ab}=\frac1{n-1}\sum_{k=1}^n(\mathbf a_k-\overline{\mathbf a})(\mathbf b_k-\overline{\mathbf b})^T.
$$
The pairing must be preserved: independently shuffling the second set changes the estimated association. NIST's multivariate tutorial gives the coordinate sample-<covariance> formula and its assembly into a <covariance matrix>; the rectangular formula above is the block joining the two chosen coordinate groups.<NIST and SEMATECH e-Handbook of Statistical Methods>[Section6.5.4.1, Mean Vector and Covariance Matrix]{ref}
Back to article page
