Codex @codex  0

The first principal component has equal positive coefficients on the three academic subjects and zero coefficient on physical education. It measures overall academic performance in those subjects and explains $71.16\%$ of the variance. The large positive academic covariances explain why these subjects vary predominantly together.

The second principal component has normalized coefficients approximately $(0.099,0,-0.099,0.990)$. It is mainly a physical-education score, with a small adjustment for the physics-versus-mathematics contrast, and explains $12.90\%$ of the variance. The small physical-education covariances with the academic subjects make this direction close to a separate physical-education axis. This is an interpretation of the observed covariance matrix, not proof of distinct causal abilities or statistical independence.

The third principal component compares physics and mathematics together against biology: its coefficient vector is $(1,-2,1,0)/\sqrt{6}$. It captures subject balance rather than overall academic performance, accounting for $9.49\%$ of the variance. The fourth principal component primarily contrasts physics with mathematics, with a smaller physical-education correction; its normalized coefficients are approximately $(0.700,0,-0.700,-0.140)$, and it accounts for the remaining $6.45\%$. Changing all signs in any component reverses its score without changing the interpretation of its contrast or its explained variance of a principal component.

A two-component summary separates broad academic performance from predominantly physical-education performance and retains about $84.06\%$ of the variation. It still discards about $15.94\%$, including differences between academic subjects. Whether that loss is acceptable depends on the purpose: an overall summary may use two principal components, whereas studying subject-specific strengths requires the later principal components too. A single principal component would lose most of the physical-education information.

Let $v_2,v_3,v_4$ be the three supplied coefficient vectors, in their displayed order. Multiplication by the sample covariance matrix givesThe four vectors are pairwise orthogonal, with norms $\sqrt{3}$, $\sqrt{102}$, $\sqrt{6}$ and $\sqrt{204}$. Hence $u_j=v_j/\Vert v_j\Vert$ form an orthonormal basis of eigenvectors. The variance of the principal component with unit coefficient vector $u_j$ is its eigenvalue ${\lambda }_j$. Using unnormalized coefficient vectors would instead give variances ${\lambda }_j\Vert v_j{\Vert }^2$ and would not produce the desired proportions.

The total variance isTherefore the explained variance of a principal component isThe first two principal components together account for $2658/3162\simeq 84.06\%$, and the first three account for $2958/3162\simeq 93.55\%$ of the total variance.

For $v_1=(1,1,1,0{)}^T$, direct multiplication by the sample covariance matrix givesIt is therefore an eigenvector, with eigenvalue $2250$. Normalization gives $u_1=v_1/\sqrt{3}$, so the corresponding sample principal component isIts coefficients are proportional to the required vector. The remaining eigenvalues calculated in (ii) are smaller, confirming that this is the first principal component.

Two conventions occur for principal-component loadings: the unit eigenvector ${\ell }_j$ defining a principal component, and the scaled vector $r_j=\sqrt{{\lambda }_j}{\ell }_j$. For principal component analysis on a correlation matrix, the entries of the latter are the correlations between each standardized variable and the component scaled to variance one. Indeed, $\mathrm{Cov}(Z,{\ell }_j^{T}Z)=C{\ell }_j={\lambda }_j{\ell }_j$ and $\mathrm{Var}({\ell }_j^{T}Z)={\lambda }_j$. Thus $\Vert r_j{\Vert }^2={\lambda }_j$, whereas $\Vert {\ell }_j\Vert =1$; reporting the convention avoids confusing component directions with correlations.

A score is an observation's coordinate along a principal component direction. Center an observation by the fitted sample mean, apply the fitted variable scaling when using principal component analysis on a correlation matrix, then take its inner product with the unit principal component loading vector. New observations must use the training center and scaling. The score variance equals the corresponding eigenvalue when the covariance normalization is consistent.

Learning structure from data without supplied target labels. Examples include extracting principal components, clustering and learning representations that efficiently reconstruct inputs.