Multivariate analysis of variance

Multivariate analysis of variance compares mean vectors of several groups while accounting for correlations between their response variables. In the one-way normal model, independent observations $Y_{ri}\sim N_p({\mu }_r,\Sigma )$ share a positive-definite covariance matrix. Put $E={\sum }_{r,i}(Y_{ri}-{\bar{Y}}_r)(Y_{ri}-{\bar{Y}}_r{)}^T$ and $H={\sum }_r{n}_r({\bar{Y}}_r-\bar{Y})({\bar{Y}}_r-\bar{Y}{)}^T$. Under equal means, these matrices have independent Wishart distributions with degrees of freedom $N-g$ and $g-1$, respectively. This follows by separating the Gaussian data into orthogonal within-group and between-group projections. The Wilks lambda statistic tests equality of the means using both matrices, rather than independently testing each coordinate.

For groups with sizes $n_r$, means ${\bar{x}}_r$, and overall mean $\bar{x}$, define $W={\sum }_{r,i}(x_{ri}-{\bar{x}}_r)(x_{ri}-{\bar{x}}_r{)}^T$ and $B={\sum }_r{n}_r({\bar{x}}_r-\bar{x})({\bar{x}}_r-\bar{x}{)}^T$. Expanding $x_{ri}-\bar{x}=(x_{ri}-{\bar{x}}_r)+({\bar{x}}_r-\bar{x})$ gives total scatter $T=W+B$ because each within-group residual sum vanishes. Both matrices are positive semidefinite, and $\mathrm{rank}B\le \min (p,g-1)$. They separate within-group noise from between-group mean separation.

For nonsingular within-group scatter $E$ and between-group scatter $H$, the statistic is ${\prod }_j(1+{\theta }_j{)}^{-1}$, where ${\theta }_j$ solve the generalized eigenvalue problem $Hv=\theta Ev$. It is small when the group means differ strongly relative to within-group variation. Maximizing a common-covariance normal likelihood under equal and unrestricted means gives the likelihood ratio ${\Lambda }^{N/2}$. Its determinant factors cancel under any nonsingular change of response coordinates. For one response it is equivalent to the analysis of variance F-test.