Past exam of the mathematics course of the University of Oxford / 2023 / Prelims / Mathematics III / 9 / b

Principal component analysis finds orthogonal directions of greatest sample variation, ordered by the variance explained. It can summarize correlated variables, reveal lower-dimensional structure, and provide a reduced representation by retaining only the first few principal components. Centering is standard; scaling may also be appropriate when the variables have incomparable units.

Let $X_c$ be the column-centered data matrix. Its sample covariance matrix is $S=X_c^T{X}_c/(n-1)$. The loadings matrix $V=(v_1,\dots ,v_p)$ has orthonormal eigenvectors of $S$ as columns, ordered by nonincreasing eigenvalues:Each principal component loading specifies the coefficients of a principal direction in the original variables. Signs are arbitrary, and equal eigenvalues permit orthogonal changes of basis within their eigenspaces.

The principal component scores are the projections of centered observations onto the loading directions:If the symbol $X$ already denotes centered data, this simplifies to $Z=XV$. The principal component loadings are directions; the principal component scores are coordinates of individual observations in those directions.