Codex @codex  0

The communality is the variance of the shared-factor contribution to variable $i$. In an orthogonal factor model, $\mathrm{Var}(X_i)=h_i^2+{\psi }_i$, where ${\psi }_i$ is its specific variance. For standardized variables, $h_i^2\le 1$. It measures variance shared through the modeled factors, not variance in a chosen principal component.

Principal component analysis replaces correlated measurements by orthogonal linear coordinates ordered by their variance. For centered observations collected in an $n\times p$ matrix $Z$, write the sample covariance matrix as $S=Z^{T}Z/(n-1)$. A unit loading vector $a$ gives score vector $Za$ with sample variance $a^{T}Sa$. The Rayleigh quotient is maximized by a unit eigenvector of $S$ for its largest eigenvalue.

The subsequent principal components maximize the same quadratic form subject to orthogonality to the earlier loading vectors. By the spectral theorem, if $S=Q\Lambda Q^T$ with descending eigenvalues, the scores $ZQ$ have diagonal covariance $\Lambda$. Their fractions of explained variance are ${\lambda }_j/{\sum }_i{\lambda }_i$. Repeated eigenvalues identify an eigenspace rather than a unique axis.

Keeping the first $q$ principal components gives the rank-$q$ reconstruction $\hat{Z}=Z{Q}_q{Q}_q^T$. Its squared reconstruction error isFor any rank-$q$ orthogonal projection $P$, retained variance is $\mathrm{tr}(SP)={\sum }_j{\lambda }_j{q}_j^{T}P{q}_j$. The weights $q_j^{T}P{q}_j$ lie in $[0,1]$ and sum to $q$, so this is at most the sum of the largest $q$ eigenvalues. Projecting onto any candidate reconstruction subspace is its best least-squares reconstruction; this proves the minimum-error property and agrees with the singular value decomposition. Thus PCA finds the linear subspace preserving the greatest variance, equivalently minimizing squared reconstruction error. The upper-left sketch shows the first axis aligned with the elongated cloud; the second is perpendicular. A scree plot and substantive interpretability help choose $q$.

These methods depend on measurement scale. A variable measured in large numerical units can dominate covariance-based principal component analysis; standardizing variables gives principal component analysis on a correlation matrix. Scores summarize patterns, but the method neither establishes causation nor guarantees that the largest-variance directions best predict a separate response.

The calculation is principal component analysis on a correlation matrix, rather than on an unscaled covariance matrix. For each of the eleven rating variables, subtract its column mean and divide by its column standard deviation. If $Z$ is the resulting $43\times 11$ matrix, use the divisor convention of the fitted object to writeHere the scaling standard deviations use divisor $43$, consistently with this convention. Using sample standard deviations with divisor $42$ gives the same correlation matrix and directions, with the corresponding score scale. Every diagonal entry of $R$ is one, so ${\sum }_k{\lambda }_k=\mathrm{tr}R=11$.

A principal component is the linear combination of the standardized ratings along $a_k$. Its principal component score for judge $i$ is $t_{ik}=z_i^T{a}_k$, and its variance is ${\lambda }_k$. The displayed component standard deviations are $\sqrt{{\lambda }_k}$; the explained variance of a principal component is ${\lambda }_k/11$. In particular,The first component explains about $92.122\%$ of the standardized variation, and the first two jointly explain $95.982\%$. Components three, four and five add about $2.321\%$, $0.830\%$ and $0.339\%$. The cumulative line adds these proportions successively; it is not the proportion explained by that component alone. Retaining two components discards about $4.018\%$ of the total standardized squared variation.

The displayed principal component loadings are the unit eigenvector coefficients $a_{jk}$. The first direction has positive, broadly similar weights for all ratings. It measures an overall favorable-rating tendency: a judge high on most ratings has a high first score. The second direction compares integrity and demeanour, with large positive coefficients, against such attributes as decision promptness, case-flow management and physical ability, with negative coefficients. Thus among similarly rated judges, a positive second score indicates relatively stronger integrity/demeanour compared with those operational ratings. This is a description of a data direction, not a causal interpretation. Multiplying a whole eigenvector and its scores by minus one reflects the display but changes nothing substantive.

These principal component loadings are not themselves variable-component correlations. With standardized variables,For example, the integrity correlation with component one is approximately $3.1833\times 0.289\simeq 0.920$. Blank loadings in the printed second column indicate suppressed small entries, not exact zeros. The complete unrounded loading vectors in the fitted object must be used to calculate scores.

A two-dimensional representation plots the 43 pairswith each point labeled by its judge. In the fitted software object these are the first two columns of the score matrix. Nearby points have similar projections of their standardized ratings. The rank-two reconstruction ${\hat{z}}_i=t_{i1}{a}_1+t_{i2}{a}_2$ is the orthogonal projection onto the leading principal plane and minimizes the total squared reconstruction error over all two-dimensional linear subspaces. To see this, for any orthonormal pair of directions the captured variance is the sum of its two Rayleigh quotients, at most ${\lambda }_1+{\lambda }_2$ by diagonalizing $R$; the remaining variance is the total minus that captured sum. Distances in the plot omit the smaller components, so a close projected pair need not be equally close in every original rating. The actual 43 coordinates cannot be recovered from the summary alone; the supplied data matrix or fitted score matrix is needed.

The printed descriptive list omits INTG, although the loading table includes it. INTG denotes integrity and supplies the eleventh variable. The standard data set also contains a contacts variable; the stated eleven-rating analysis uses the ratings, not contacts.

The command centres the log measurements and performs principal component analysis of their covariance matrix. With cor=F, it does not first standardize the variables to variance one. For a unit principal component loading vector $q$, the score $q^T(z_i-\bar{z})$ has sample variance $q^T\hat{\Sigma }q$. By the spectral theorem for real symmetric matrices, expand $q$ in an orthonormal eigenbasis; that variance is a weighted average of the eigenvalues. Its maximum is the largest eigenvalue, reached at the corresponding eigenvector. Repeating subject to orthogonality to previous directions gives the other principal components.

The software uses the divisor-$31$ sample covariance matrix for its component standard deviations, whereas var used divisor $30$. Diagonalizing the displayed matrix gives approximatelyTaking square roots of the latter gives $(0.5671603,0.07312405,0.03064834)$, agreeing with the output to the rounding of the displayed sample covariance matrix. The normalization changes these numerical standard deviations but leaves the eigenvectors and proportions of explained variance of a principal component unchanged.

The first principal component explains about $98.08\%$ of the total log-variable variance, the second another $1.63\%$, and the last about $0.286\%$. Keeping two preserves about $99.714\%$. An original scree plot displays the sharp drop after the first component:

Consequently the centered cloud is very elongated along one direction: a one-dimensional approximation represents almost all of its squared variation. From the displayed covariance matrix, a unit first eigenvector, with its arbitrary sign chosen positive, is approximatelyIts principal component score is therefore $0.3980(\log G_i-\overline{\log G})+0.09514(\log H_i-\overline{\log H})+0.9124(\log V_i-\overline{\log V})$. All coefficients have the same sign, so this direction describes overall tree size, with log volume dominating the unstandardized variation. The summary alone supplies proportions, not the loading directions; the displayed sample covariance matrix supplies the latter. This is a geometric approximation, not evidence of an exact deterministic relation or independent original variables.

One further use is principal component analysis on a correlation matrix, obtained with cor=T, which equalizes marginal variances and answers a different scaling-sensitive question. Another is principal component regression: fit a response to selected principal component scores rather than highly correlated original predictors, choosing the number using prediction assessment. The fitted transformation also projects new measurements into the same low-dimensional coordinates, provided the training centering and scaling are retained.

Take logarithms entry by entry before centering. For tree $i$, let $z_i=(\log G_i,\log H_i,\log V_i{)}^T$, and put $\bar{z}=31^{-1}{\sum }_i{z}_i$. The command var(log(b)) computes the sample covariance matrix with the unbiased estimator normalizationThus entry $(j,k)$ is the sum of products of the centered log measurements divided by $30$, not the logarithm of the covariance between the original measurements. Its diagonal entries are the three log-variable sample variances, and the off-diagonal entries are their sample covariances. The positive off-diagonal entries indicate positive association of the log measurements. Taking logarithms turns a multiplicative relation such as $V\approx c{G}^{2}H$ into the approximately linear relation $\log V\approx \log c+2\log G+\log H$, which motivates investigating principal components of these transformed observations.

For the sample, put $\bar{x}=n^{-1}{\sum }_i{x}_i$ and use the sample covariance matrixOrder its eigenvalues ${\hat{\lambda }}_j$ decreasingly and choose unit orthogonal eigenvectors ${\hat{\ell }}_j$. The sample principal components are the score columnsTheir sample means are zero and their sample covariance is diagonal, since ${\hat{\ell }}_j^{T}S{\hat{\ell }}_k={\hat{\lambda }}_j{1}_{\{j=k\}}$. The $j$th sample variance is ${\hat{\lambda }}_j$, and its explained variance of a principal component is ${\hat{\lambda }}_j/\mathrm{tr}(S)$. Using divisor $n$ instead changes the numerical variances by a common factor but leaves the component directions unchanged.

Standardizing means replacing a variable with positive sample variance by $(x_{ij}-{\bar{x}}_j)/s_j$. This gives principal component analysis on a correlation matrix, using $C=D^{-1/2}S{D}^{-1/2}$ with $D=\mathrm{diag}(S)$. It removes arbitrary differences of measurement units and prevents a large-variance coordinate from dominating solely through its scale. It also changes the question being optimized: small-variance and possibly noisy coordinates receive equal marginal weight. Principal component analysis based on the covariance matrix can be more appropriate when the variables have common meaningful units and absolute variability matters. Standardization is therefore a substantive choice, not an automatic improvement. A zero-variance variable must first be removed. In the word-rating example the variables share the same numerical range, but their across-word dispersions can differ; standardization gives each attribute equal initial variance.

For the standardized eight-variable analysis, total variance is $\mathrm{tr}(C)=8$. The first three fractions of explained variance of a principal component are approximately 59.6%, 19.1% and 10.1%, respectively. The first two account for about 78.8%, and the first three for 88.9%. The first direction is dominant but one direction alone loses substantial variation. Retaining two principal components is a plausible descriptive summary: their eigenvalues exceed $1$, whereas the third is below $1$. That cutoff is a heuristic, and the third principal component may still have interpretable content.

The reported coefficient rows are not unit-length vectors. The printed rounded rows have squared lengths $4.7514$, $1.5222$ and $0.8063$, consistent with their eigenvalues to the precision of the coefficient table. They are consistent with the scaled principal component loading convention $r_j=\sqrt{{\lambda }_j}{\ell }_j$. Such rows remain eigenvectors, but the score direction with unit length is ${\ell }_j=r_j/\sqrt{{\lambda }_j}$, or more accurately the row divided by its actual Euclidean norm when using rounded entries. Under this convention $r_{ij}$ is the correlation of standardized attribute $i$ with the variance-one score for component $j$:This explains both their sizes and why they must not be treated as the unit vectors from parts (i) and (ii).

The first principal component has positive coefficients on every attribute, strongest on friendliness, goodness, niceness, bravery and strength. With the favorable endpoints coded in the positive direction, it describes a broad favorable or impressive evaluation, with a potency contribution. The second contrasts movement and speed, with some size and strength, against the first three evaluative attributes; it is chiefly an activity dimension. The third is dominated by size, with smaller positive strength and negative movement/speed coefficients, distinguishing size or potency from activity. These are interpretations of directions of variation, not evidence of causal latent traits. Reversing a principal component's overall sign changes neither its meaning as an axis nor its explained variance; endpoint coding fixes only how its positive side is described.

Factor analysis models a large set of correlated measurements using a smaller collection of shared latent factors and variable-specific variation. A classical orthogonal factor model isTake $q<p$, centered errors, and positive specific variances for the inverse-covariance and likelihood formulas below. The factor loadings form the $p\times q$ matrix $L$, and the model covariance isThus for $i\ne k$, $\mathrm{Cov}(X_i,X_k)={\sum }_j{L}_{ij}{L}_{kj}$; the common latent factors explain off-diagonal correlation. The communality of variable $i$ is $h_i^2={\sum }_j{L}_{ij}^2$, and $\mathrm{Var}(X_i)=h_i^2+{\psi }_i$. For standardized variables this is $1=h_i^2+{\psi }_i$. The specific variance represents measurement noise or genuinely unshared variation; it is not another common latent factor.

This differs from principal component analysis. A principal component maximizes projected total variance and is a determined linear score after choosing its direction; a factor model attempts to explain shared covariance while reserving diagonal specific variance. Low-dimensional PCA truncation can help initialize a factor fit, but it does not justify setting all specific variances to zero. Latent factors are unobserved, not directly observed scores. When the latent factors and the errors are jointly Gaussian, their conditional mean iswhich supplies one factor-score estimate, with remaining uncertainty. This follows from the block conditional normal formula in Question 1; it does not make the latent factor uniquely recoverable from one observation.

For the Gaussian likelihood, use the empirical covariance $S_0=n^{-1}{\sum }_i(x_i-\bar{x})(x_i-\bar{x}{)}^T$. Fit the factor model by minimizing $\log |L{L}^T+\Psi |+\mathrm{tr}(S_0(L{L}^T+\Psi {)}^{-1})$ subject to admissible specific variances. Other approaches estimate communalities and extract latent factors from the reduced correlation matrix. Choose $q$ using substantive plausibility, a scree plot, fitted residual correlations and model-fit assessment. Under local identifiability, a likelihood goodness-of-fit count for the covariance model iswhere the subtracted rotational dimension must be removed from the loading parameter count. This count alone does not establish identifiability, and boundary fits with zero specific variance need special care.

Factor rotation addresses interpretability and nonuniqueness. For every orthogonal matrix $Q$, replace $(L,F)$ by $(LQ,Q^{T}F)$; then $(LQ)(LQ{)}^T=L{L}^T$, so the fit is unchanged. The absolute orientation of unrestrained latent factors is therefore not identified by the covariance. Varimax rotation chooses orthogonal axes favoring a simple pattern of large and small squared loadings. Oblique rotation can permit correlated latent factors, provided their covariance is transformed consistently. Sign changes and permutations are further harmless relabelings; an interpretation must state its chosen convention.

The sketch uses four standardized hypothetical measurements sharing two orthogonal latent factors. Rotating the factor coordinates changes the loading descriptions while preserving every implied covariance and communality; the scree plot belongs to that same constructed covariance matrix. Factor analysis separates shared variation from specific variance, and its latent factors require an identifying or interpretive orientation.

Principal component analysis replaces the original coordinates by orthogonal linear combinations, ordered so that the first few retain as much variance as possible. It gives a lower-dimensional summary for visualization or compression and reveals the main directions of variation. A direction of large variance need not be the direction most useful for classification.

Write $\mu =\mathbb{E}X$. For a unit vector $u$, the centered score $Y=u^T(X-\mu )$ has varianceMaximizing this Rayleigh quotient with a Lagrange multiplier givesThus a maximizing direction must be an eigenvector of the covariance matrix. To identify the maximum, use the spectral theorem for real symmetric matrices to choose an orthonormal basis of eigenvectors $v_1,\dots ,v_p$, with eigenvalues ${\lambda }_1\ge \cdots \ge {\lambda }_p\ge 0$. Writing $u={\sum }_j{c}_j{v}_j$ givesEquality is attained at $u=v_1$. Consequently the first population principal component isFor each subsequent principal component, maximize the same variance over unit coefficient vectors orthogonal to all the previously chosen ones. Restricting the expansion above to the remaining eigenvectors gives $v_2$, then $v_3$, and so on. ThusThe principal components are uncorrelated; they need not be independent unless additional distributional assumptions, such as joint Gaussianity, hold. Their total variance is $\mathrm{tr}\Sigma ={\sum }_j{\lambda }_j$. Signs of eigenvectors are arbitrary, and a repeated eigenvalue permits any orthonormal basis in its eigenspace. In sample principal component calculations, replace $\mu$ by the sample mean and $\Sigma$ by the sample covariance matrix.

The first principal component has equal positive coefficients on the three academic subjects and zero coefficient on physical education. It measures overall academic performance in those subjects and explains $71.16\%$ of the variance. The large positive academic covariances explain why these subjects vary predominantly together.

The second principal component has normalized coefficients approximately $(0.099,0,-0.099,0.990)$. It is mainly a physical-education score, with a small adjustment for the physics-versus-mathematics contrast, and explains $12.90\%$ of the variance. The small physical-education covariances with the academic subjects make this direction close to a separate physical-education axis. This is an interpretation of the observed covariance matrix, not proof of distinct causal abilities or statistical independence.

The third principal component compares physics and mathematics together against biology: its coefficient vector is $(1,-2,1,0)/\sqrt{6}$. It captures subject balance rather than overall academic performance, accounting for $9.49\%$ of the variance. The fourth principal component primarily contrasts physics with mathematics, with a smaller physical-education correction; its normalized coefficients are approximately $(0.700,0,-0.700,-0.140)$, and it accounts for the remaining $6.45\%$. Changing all signs in any component reverses its score without changing the interpretation of its contrast or its explained variance of a principal component.

A two-component summary separates broad academic performance from predominantly physical-education performance and retains about $84.06\%$ of the variation. It still discards about $15.94\%$, including differences between academic subjects. Whether that loss is acceptable depends on the purpose: an overall summary may use two principal components, whereas studying subject-specific strengths requires the later principal components too. A single principal component would lose most of the physical-education information.