Past exam of the mathematical and Computational Biology course of the University of Cambridge / 2024 / Paper 1 / E8

Yes, the data are centred. Both coordinate sample means vanish:Thus principal component analysis can be applied without further data centering.

Place observations in rows of the data matrix $X$. Since the coordinate sample means are zero, the Gram matrix of its columns isThe sample covariance matrix is $S=X^{T}X/(3-1)$; using divisor three instead changes eigenvalues by a common factor and leaves principal component loadings unchanged. For $X^{T}X$, the characteristic polynomial isFor the larger eigenvalue $150$, the eigenvector equation gives $-36\alpha -48\beta =0$, so $3\alpha +4\beta =0$. Normalize to unit Euclidean norm and choose $v_1=(4/5,-3/5{)}^T$. An orthogonal unit eigenvector for $50$ is $v_2=(3/5,4/5{)}^T$. ThereforeThe sample variances of these principal components are $75$ and $25$, respectively. Their principal component scores for observations $A,B,C$ areEither eigenvector may be multiplied by $-1$ without changing its principal component direction or its variance. No coordinate standardization is applied here: that would instead be principal component analysis on a correlation matrix.

The explained variance of a principal component is its eigenvalue divided by the sum of all eigenvalues of the covariance matrix. A common divisor cancels, so the first principal component explainsof the total variance.

The one-dimensional principal component analysis reconstruction lies along $v_1=(4/5,-3/5{)}^T$ through the zero sample mean. Its discarded principal component coordinate is $P{C}_2=0$, soAt $x=6$, $\boxed{\hat{y}=-4.5}$. This intersects the retained principal component line with the vertical line at the supplied $x$. It is a reconstruction from that line, rather than an ordinary linear regression of $y$ on $x$.

The premise contains a wording error: $\lambda$ must be a nonzero eigenvalue, rather than a nonzero eigenvector. Let $v\ne 0$ satisfyThen $Xv\ne 0$, since otherwise $X^{T}Xv=0$ would force $\lambda v=0$. Multiplying by $X$ givesThus $\boxed{\lambda \text{ is an eigenvalue of }X{X}^T}$, with eigenvector $Xv$. Conversely, applying $X^T$ maps an eigenvector of $X{X}^T$ with nonzero eigenvalue to one of $X^{T}X$.

These maps are inverses up to the factor $\lambda$ on the corresponding eigenspaces, so the nonzero eigenvalues have equal multiplicities. Their zero eigenvalues need not have the same multiplicity because the two matrices can have different sizes. For a unit eigenvector $v$ and real $X$, the normalized partner is $u=Xv/\sqrt{\lambda }$, since $\Vert Xv{\Vert }^2=v^T{X}^{T}Xv=\lambda >0$.

For centred data, classical multidimensional scaling from Euclidean distances constructs the observation Gram matrixwhere $D_{ij}^{(2)}=\Vert x_i-x_j{\Vert }^2$. By part (i), its nonzero eigenvalues agree with those of $X^{T}X$, the matrix used for principal component analysis before dividing by a covariance normalization.

If $u_i=X{v}_i/\sqrt{{\lambda }_i}$ is the unit observation-space eigenvector, the corresponding classical multidimensional scaling coordinate column isThat is exactly the column of principal component scores. Consequently classical multidimensional scaling of Euclidean pairwise distances recovers the same reduced coordinates as principal component analysis, up to signs and arbitrary orthogonal transformations within equal-eigenvalue subspaces. Distances alone cannot fix translation or orientation. The equivalence presumes centred data and Euclidean distances; general dissimilarities need not yield a positive semidefinite Gram matrix or a corresponding data covariance matrix.