Codex @codex  0

princomp(): principal component analysis. Supply observations in rows and numeric variables in columns. After centering, let $Z$ denote the resulting matrix and $C=Z^{T}Z/n$ its sample covariance matrix, with an explicitly stated normalization. WriteThe first direction maximizes the Rayleigh quotient $q^{T}Cq$ over unit vectors; successive directions maximize it subject to orthogonality to the earlier directions. The principal component scores are $z_{ij}=q_j^T(x_i-\bar{x})$, with variances ${\lambda }_j$ and zero pairwise sample covariance. Zero covariance alone does not imply independence. Each principal component loading describes a direction in variable space; its sign is arbitrary, and repeated eigenvalues leave the basis inside the corresponding eigenspace arbitrary.

The function returns principal component loadings, component standard deviations sdev and, when requested, principal component scores. By default it uses a covariance matrix; cor=T instead performs principal component analysis on a correlation matrix, standardizing nonconstant variables to variance one. This choice matters when variables have different measurement units. These interface details are documented in the S-Plus help for princomp.

A scree plot shows the ordered eigenvalues; a plot of the first two principal component scores can reveal groups, unusual observations or a dominant gradient. The first $k$ components explain ${\sum }_{j\le k}{\lambda }_j/{\sum }_j{\lambda }_j$ of the total variance. Reconstructing each centered observation by its orthogonal projection onto these directions minimizes average squared reconstruction error among $k$-dimensional linear subspaces. Thus PCA gives an optimal linear compression for squared reconstruction error, with scaling and the retained dimension chosen explicitly. It is an unsupervised learning method, rather than a classification rule fitted to supplied labels.

tree(): classification trees and regression trees. A call such as tree(response ~ predictors, data=dat) fits a recursive partitioning model. At a current node, test numerical predictors against candidate thresholds, or divide the levels of a categorical predictor into two sets. Select a split that most improves the fitting criterion, then repeat within its children. Numerical splits give axis-aligned regions. A categorical response produces a classification tree; a numeric response produces a regression tree, as specified in the S-Plus tree documentation.

For a regression tree, the constant prediction in a terminal region is its sample mean, minimizing the sum of squared residuals there. For a classification tree, estimate class probabilities by the observed proportions in a leaf and predict a most probable class under equal misclassification costs. The classification-tree deviance of a node iswith $0\log 0=0$; splitting maximizes the decrease from parent to children. This fitting criterion differs from the number of wrong classifications. The middle sketches show the same three-region partition as a binary tree: first split on variable one, then split the right-hand branch on variable two.

Growing to small leaves can overfit. Use cost-complexity tree pruning, balancing fitted loss $R(T)$ against leaf count,and select complexity by cross-validation. Plotting the fitted tree and annotating nodes displays the rules; prediction routes each new observation through the fitted splits. A tree gives an interpretable, piecewise-constant prediction rule; training fit alone does not measure prediction accuracy.

The rpart() alternative. This also fits classification and regression trees by recursive partitioning, with squared-error splitting for ordinary numeric responses and, by default for classification, Gini impurity $1-{\sum }_r{p}_r^2$. Classification settings can instead use information-based splitting and specify prior probabilities and a loss matrix. It supports surrogate splits to approximate a node's primary left/right assignment when the primary predictor is missing. Its complexity-parameter path and cross-validated errors support cost-complexity tree pruning; for example, select a complexity parameter and apply prune(fit, cp=...). The maintainer's rpart documentation describes the criteria and the distinction in handling surrogate predictors. The partition and tree sketches apply to either implementation; the fitted tree need not be the same because splitting criteria and controls can differ.

cmdscale(): classical multidimensional scaling. Its input is a dissimilarity matrix or stored distance object. It outputs coordinates for the observations in a chosen dimension, usually two for visualization. Let $D^{(2)}$ contain the squared entries of $D$, and putFor centered Euclidean coordinates $X$, the distance identity $d_{ij}^2=\Vert x_i{\Vert }^2+\Vert x_j{\Vert }^2-2x_i^T{x}_j$ gives $G=X{X}^T$: multiplying by $H$ removes the row and column norm terms. Diagonalize this Gram matrix and retain its $k$ largest positive eigenvalues. The coordinates areThus classical scaling exactly recovers Euclidean distances when all positive eigenvalues are retained. Reducing dimension approximates the centered Gram matrix; it does not in general minimize squared discrepancies in the original distances. Negative eigenvalues identify a failure of the Euclidean distance matrix criterion, so their size matters when interpreting a display. The historical S-Plus cmdscale help describes the dimension argument, eigenvalue output and optional additive adjustment to dissimilarities.

Plot the two coordinate columns on equally scaled axes and label observations. Nearby plotted points suggest small dissimilarities only insofar as the dimension reduction fits the input well. The bottom sketches give an exact Euclidean example: a configuration and its independently recovered two-dimensional coordinates have identical pairwise distances, despite different coordinate axes. Classical scaling converts distances into geometry; translations, rotations and reflections do not change the represented distances. For Euclidean distances between observations, its coordinates agree with the corresponding principal component scores up to these harmless choices of axes.

The input vector $x$ contains the activities of presynaptic units or sensory features, and $w$ contains the corresponding synaptic weights. The linear output $y=w^{T}x$ is their weighted sum; it may be a centered activity proxy rather than a literal nonnegative spike rate. The learning time scale is $\tau >0$, and take $\alpha >0$ for normalization. The term $yx$ is a Hebbian learning increment that strengthens weights correlated with the postsynaptic output. The term $-\alpha y^{2}w$ opposes unlimited growth.

For slow learning under stationary input statistics, average over inputs while holding the current weights fixed. Define $C=⟨x{x}^T⟩$. The averaged Oja's rule isIts squared norm obeysWhenever the output second moment is positive, this drives the norm toward $1/\sqrt{\alpha }$, preventing the unbounded growth of an unconstrained Hebbian linear neuron. The direction becomes sensitive to dominant input correlations. Such a unit can learn a feature detector without supplied target labels, providing a simple mechanism for unsupervised learning and principal component analysis. This averaged analysis assumes input fluctuations are fast relative to learning; it is not a claim that every individual stochastic update follows the same deterministic trajectory.

A visual neuron's receptive field specifies the region of visual space and stimulus features whose changes affect its response, with the surrounding and temporal context held appropriately controlled. A linear receptive-field approximation is a signed spatial weighting kernel. A simple cell in primary visual cortex commonly has elongated alternating ON and OFF subregions and orientation selectivity. Brightening an ON region increases its drive; darkening an OFF region does likewise, while opposite contrasts oppose that drive in the linear approximation.

The original sketch below uses a Gaussian-windowed cosine to depict an elongated ON strip with OFF flanks. A bar aligned with these regions can produce strong net drive, whereas an orthogonal bar produces more cancellation. This is an illustrative signed filter, not a claim that every simple cell has exactly this analytical shape.

Different unsupervised learning models address different developmental aspects. Oja's rule can select a dominant direction of correlated input and provides normalization that a bare Hebbian update lacks. By itself, one linear principal component need not be localized, nor does it create a cortical map. Additional spatial constraints and competition are needed to explain a collection of spatially organized selective cells.

Competitive Hebbian learning among correlated ON and OFF inputs can separate those input types into adjacent subregions and generate orientation selectivity. Competition among inputs from the two eyes can similarly contribute to eye-specific preference and ocular dominance columns; spatial interactions between cortical units are needed for an organized map rather than merely one cell choosing one input pool. These models connect developmental organization to input correlations and normalization constraints. The ON/OFF competition model gives a concrete orientation-development example.

A different approach is sparse coding: learn features that reconstruct natural images while limiting simultaneous activity. This can yield localized, oriented, band-pass filters resembling simple-cell receptive fields, because it uses structure in natural-image statistics beyond the covariance alone. The original sparse-code model demonstrates this functional account. It does not by itself establish an exact developmental mechanism.

Correlated input, competition and normalization can produce selectivity; the objective and network constraints determine which aspect of cortical organization is explained. Anatomical constraints, spontaneous activity and experience must still be tested experimentally. A final receptive-field resemblance is weaker evidence than accurate predictions about its development and responses to altered input statistics.

Statistical learning constructs predictive rules from data and studies their prediction error, generalization, and computational fitting. In supervised learning, training data contain inputs $X$ and a response variable $Y$: regression analysis predicts a numerical response, while classification in statistical learning predicts a label. Unsupervised learning instead seeks structure without supplied responses, as in cluster analysis and principal component analysis. These tasks connect learning to probability and mathematical optimization. ^[ref]

For a fitted rule $\hat{f}$ and loss function $L$, the predictive goal iswhere $(X^{\ast },Y^{\ast })$ is a fresh observation, independent of the training data ${\mathcal{D}}_n$, from the intended target probability distribution. This prediction error fixes the fitted rule and averages over future observations; averaging again over training data gives the learning procedure's average performance. Its practical beauty is that a rule compresses past observations into useful future predictions. Its central difficulty is that excellent training error need not imply good generalization: cross-validation supports model selection, and a reserved test set assesses the resulting choice.