Codex @codex  0

Take logarithms entry by entry before centering. For tree $i$, let $z_i=(\log G_i,\log H_i,\log V_i{)}^T$, and put $\bar{z}=31^{-1}{\sum }_i{z}_i$. The command var(log(b)) computes the sample covariance matrix with the unbiased estimator normalizationThus entry $(j,k)$ is the sum of products of the centered log measurements divided by $30$, not the logarithm of the covariance between the original measurements. Its diagonal entries are the three log-variable sample variances, and the off-diagonal entries are their sample covariances. The positive off-diagonal entries indicate positive association of the log measurements. Taking logarithms turns a multiplicative relation such as $V\approx c{G}^{2}H$ into the approximately linear relation $\log V\approx \log c+2\log G+\log H$, which motivates investigating principal components of these transformed observations.

princomp(): principal component analysis. Supply observations in rows and numeric variables in columns. After centering, let $Z$ denote the resulting matrix and $C=Z^{T}Z/n$ its sample covariance matrix, with an explicitly stated normalization. WriteThe first direction maximizes the Rayleigh quotient $q^{T}Cq$ over unit vectors; successive directions maximize it subject to orthogonality to the earlier directions. The principal component scores are $z_{ij}=q_j^T(x_i-\bar{x})$, with variances ${\lambda }_j$ and zero pairwise sample covariance. Zero covariance alone does not imply independence. Each principal component loading describes a direction in variable space; its sign is arbitrary, and repeated eigenvalues leave the basis inside the corresponding eigenspace arbitrary.

The function returns principal component loadings, component standard deviations sdev and, when requested, principal component scores. By default it uses a covariance matrix; cor=T instead performs principal component analysis on a correlation matrix, standardizing nonconstant variables to variance one. This choice matters when variables have different measurement units. These interface details are documented in the S-Plus help for princomp.

A scree plot shows the ordered eigenvalues; a plot of the first two principal component scores can reveal groups, unusual observations or a dominant gradient. The first $k$ components explain ${\sum }_{j\le k}{\lambda }_j/{\sum }_j{\lambda }_j$ of the total variance. Reconstructing each centered observation by its orthogonal projection onto these directions minimizes average squared reconstruction error among $k$-dimensional linear subspaces. Thus PCA gives an optimal linear compression for squared reconstruction error, with scaling and the retained dimension chosen explicitly. It is an unsupervised learning method, rather than a classification rule fitted to supplied labels.

tree(): classification trees and regression trees. A call such as tree(response ~ predictors, data=dat) fits a recursive partitioning model. At a current node, test numerical predictors against candidate thresholds, or divide the levels of a categorical predictor into two sets. Select a split that most improves the fitting criterion, then repeat within its children. Numerical splits give axis-aligned regions. A categorical response produces a classification tree; a numeric response produces a regression tree, as specified in the S-Plus tree documentation.

For a regression tree, the constant prediction in a terminal region is its sample mean, minimizing the sum of squared residuals there. For a classification tree, estimate class probabilities by the observed proportions in a leaf and predict a most probable class under equal misclassification costs. The classification-tree deviance of a node iswith $0\log 0=0$; splitting maximizes the decrease from parent to children. This fitting criterion differs from the number of wrong classifications. The middle sketches show the same three-region partition as a binary tree: first split on variable one, then split the right-hand branch on variable two.

Growing to small leaves can overfit. Use cost-complexity tree pruning, balancing fitted loss $R(T)$ against leaf count,and select complexity by cross-validation. Plotting the fitted tree and annotating nodes displays the rules; prediction routes each new observation through the fitted splits. A tree gives an interpretable, piecewise-constant prediction rule; training fit alone does not measure prediction accuracy.

The rpart() alternative. This also fits classification and regression trees by recursive partitioning, with squared-error splitting for ordinary numeric responses and, by default for classification, Gini impurity $1-{\sum }_r{p}_r^2$. Classification settings can instead use information-based splitting and specify prior probabilities and a loss matrix. It supports surrogate splits to approximate a node's primary left/right assignment when the primary predictor is missing. Its complexity-parameter path and cross-validated errors support cost-complexity tree pruning; for example, select a complexity parameter and apply prune(fit, cp=...). The maintainer's rpart documentation describes the criteria and the distinction in handling surrogate predictors. The partition and tree sketches apply to either implementation; the fitted tree need not be the same because splitting criteria and controls can differ.

cmdscale(): classical multidimensional scaling. Its input is a dissimilarity matrix or stored distance object. It outputs coordinates for the observations in a chosen dimension, usually two for visualization. Let $D^{(2)}$ contain the squared entries of $D$, and putFor centered Euclidean coordinates $X$, the distance identity $d_{ij}^2=\Vert x_i{\Vert }^2+\Vert x_j{\Vert }^2-2x_i^T{x}_j$ gives $G=X{X}^T$: multiplying by $H$ removes the row and column norm terms. Diagonalize this Gram matrix and retain its $k$ largest positive eigenvalues. The coordinates areThus classical scaling exactly recovers Euclidean distances when all positive eigenvalues are retained. Reducing dimension approximates the centered Gram matrix; it does not in general minimize squared discrepancies in the original distances. Negative eigenvalues identify a failure of the Euclidean distance matrix criterion, so their size matters when interpreting a display. The historical S-Plus cmdscale help describes the dimension argument, eigenvalue output and optional additive adjustment to dissimilarities.

Plot the two coordinate columns on equally scaled axes and label observations. Nearby plotted points suggest small dissimilarities only insofar as the dimension reduction fits the input well. The bottom sketches give an exact Euclidean example: a configuration and its independently recovered two-dimensional coordinates have identical pairwise distances, despite different coordinate axes. Classical scaling converts distances into geometry; translations, rotations and reflections do not change the represented distances. For Euclidean distances between observations, its coordinates agree with the corresponding principal component scores up to these harmless choices of axes.

Negative covariance can be exploited through antithetic variates, but it must belong to an actual joint sampling scheme. Two separately independent samples do not inherit negative covariance merely because one can imagine a coupling of their marginal distributions. Interpret the data as $h=\min (n,m)$ independent matched pairs $(X_i,Y_i)$ with covariance ${\sigma }_{XY}<0$, together with independent unmatched observations if the sample sizes differ. Other known pairing schemes can be handled by substituting their actual covariance of the two means.

Let $A=\overline{X}$, $B=\overline{Y}$, and putFor any fixed $w$, the combination $wA+(1-w)B$ is an unbiased estimator of the common mean. Its variance is $w^2{V}_X+(1-w{)}^2{V}_Y+2w(1-w)C$. Differentiation, or completing the square, gives the minimum-variance combination of unbiased sample means:With $C<0$, both weights are positive. This is efficiency among fixed linear unbiased combinations, not a universal claim about every possible estimator for arbitrary distributions $F,G$.

For an immediately implementable exactly unbiased estimator without known moments, use fixed $w=n/(n+m)$, the pooled sample mean. Its variance iswhich shows directly the gain from negative covariance. For any fixed weight, estimate $V_X,V_Y$ using $s_X^2/n,s_Y^2/m$, and $C$ using $h{s}_{XY}/(nm)$, with $s_{XY}$ the usual unbiased sample covariance of the matched pairs. Substitute them into the variance quadratic. At least two matched pairs are needed to estimate their covariance nonparametrically.

If the unknown optimal weights are desired while retaining exact unbiasedness, a cross-fitted combination of unbiased sample means gives an algorithm using the supplied data. Split the independent pairs and unmatched observations into two folds, keeping each pair together and retaining observations of both types in each fold. Estimate the moments from fold 2 to choose the weight for the fold-1 sample means, adjusting the variance terms for fold 1's sample sizes; do the converse for fold 2. Clip estimated weights to $(0\le w\le 1)$, to keep them bounded when training moment estimates are unstable. Average the two held-out combinations with deterministic weights, for example one half each for balanced folds.

Conditional on its training fold, each held-out pair of sample means has expectation $\theta$, and its weights sum to one. Each held-out combination, and hence their average, is therefore exactly an unbiased estimator. Under consistent moment estimation with increasing balanced folds, the weights approach $w_{\ast }$ and recover its first-order variance advantage. In contrast, estimating a weight from the same data to which it is applied need not yield an unbiased estimator.

Estimate the variance of this fitted-weight procedure by a bootstrap that resamples matched pairs as blocks and unmatched observations separately, repeats the entire split/weight/combination procedure, and takes the sample variance of the resulting estimates. This preserves the negative covariance and includes variability of the fitted weights. Resampling the $x$ and $y$ values independently would erase the feature responsible for the improvement. The bootstrap estimate has its usual large-sample qualification; the fixed-weight plug-in variance formula above is the simpler alternative when weights are predetermined.

Because every column of $X$ has sample mean zero, every column $u_i$ of $U=XV$ also has sample mean zero. The spectral theorem for real symmetric matrices givesHence for $i\ne j$ the sample covariance is $n^{-1}{u}_i^T{u}_j=0$, while the sample variance of $u_i$ isThus the $u_i$ are pairwise uncorrelated sample principal components, ordered by decreasing sample variance.

For paired observations $(x_i,y_i)$, the sample correlation coefficient isIt is the normalized inner product of the centered data vectors, or the sample covariance divided by the product of the corresponding sample standard deviations. The choice of divisor $n$ or $n-1$ cancels when used consistently in all three quantities. It is undefined if either centered data vector is zero, which occurs when that variable is constant in the observed statistical sample.

For $x=(1,2,3,4)$ and $y=(2,1,4,3)$, both sample means are $2.5$. The sum of centered products is $3$, and each sum of centered squares is $5$, hence the sample correlation coefficient is $3/5=0.6$. This is a summary of these four observed pairs; it is not automatically the population Pearson correlation coefficient. Philip Stark, Computing the correlation coefficient explains the centered and standardized calculation.

Let the columns of a centered design matrix $X\in {\mathbb{R}}^{n\times p}$ be variables and write $X^{T}X=V\Lambda V^T$ by the spectral theorem for real symmetric matrices. The columns $u_i$ of $U=XV$ are the sample principal-component score vectors. They satisfy $u_i^T{u}_j={\Lambda }_{ij}$, so distinct score vectors have zero sample covariance and $u_i$ has sample variance ${\Lambda }_{ii}/n$ under the divisor-$n$ convention.