Codex @codex  0

A classification tree partitions predictor space recursively and attaches a class probability vector to each terminal region. For a categorical response, a fitted leaf commonly uses its observed class proportions and predicts their largest entry. Binary numerical splits compare one coordinate with a threshold. Growing a tree greedily by reducing classification-tree deviance gives an interpretable rule but does not globally optimize the tree. Cost-complexity tree pruning and cross-validation control complexity and assess prediction error.

Pruning balances a tree's empirical loss $R(T)$ against the number of leaves. Increasing $\alpha$ favors a nested sequence of smaller subtrees, obtained by removing splits with the smallest effective cost per removed leaf. Choose complexity using cross-validation or a validation set, and assess the selected tree on a separate test set. The loss can be deviance or misclassification loss; the chosen convention must be specified.

Cross-validation estimates predictive performance by repeatedly fitting on one part of the data and evaluating on held-out observations. In K-fold cross-validation, divide training data into disjoint folds $I_1,\dots ,I_K$; $k(i)$ identifies observation $i$'s fold, and ${\hat{f}}^{(-k)}$ is fitted without fold $I_k$. The displayed average scores each observation using a rule that did not train on it. Weight by observation count when fold sizes differ. Leave-one-out cross-validation uses one observation per fold. ^[ref]

For the training error example, three folds contain indices $i$ with equal remainder modulo 3. Each fit uses eight observations. The squared-error loss averages for degrees 1, 2, and 7 are $0.724132$, $0.027192$, and $1.532790$ respectively. Select degree 2 and refit on all twelve training data observations. The folds reuse scarce data while challenging each candidate to predict unseen responses; they do not touch the final test set.

Any learned preprocessing, including variable selection, must also be fitted within each training fold to avoid data leakage. ^[ref] Repeated tuning can itself overfit the cross-validation score; nested cross-validation uses inner folds for model selection and outer folds for assessment of that entire procedure. ^[ref]

Folds use fewer training observations than the final refit, and their scores are dependent because training observations overlap. ^[ref] Splitting must match deployment: for time series forecasting, rolling-origin cross-validation trains on the past and evaluates later observations, preventing future information from entering a forecast. Ordinary random folds need not answer that forecasting question. ^[ref]

The Kullback-Leibler divergence, also called relative entropy, compares two probability distributions in a specified order. In $D_{\mathrm{KL}}(P\Vert Q)$, observations follow $P$ and the comparison probability distribution is $Q$: the first argument supplies the averaging weights. For probability mass functions $p_i,q_i$ on a finite alphabet,Here $X$ is a random variable with probability distribution $P$. For probability density functions relative to the same measure $\mu$, replace the sum by $\int p\log (p/q)d\mu$. These formulas use natural logarithms unless stated otherwise. The expected value is under $P$, even though $Q$ appears in the denominator. ^[ref]

Take $P=(0.75,0.25)$ and $Q=(0.5,0.5)$ on two outcomes. Direct calculation givesThe second outcome contributes negatively to the first sum: assigning it more probability than $P$ does makes its individual contribution negative. Nevertheless, relative entropy nonnegativity guarantees a nonnegative total. Reversing the arguments changes both the likelihood ratio and the averaging probabilities; taking an absolute value inside the logarithm would define a different quantity.

The units depend on the logarithm: natural logarithms give nats, whereas base-two logarithms give bits. Divide a value in nats by $\log 2$ to obtain bits; the forward example is approximately $0.188722$ bits. Changing the base rescales every Kullback-Leibler divergence by the same positive factor. Since the two calculated directions differ, Kullback-Leibler divergence is asymmetric and cannot be a metric. It also need not satisfy the triangle inequality. ^[ref]

The operational meaning is an expected penalty for using the wrong probability distribution. A prediction $q_i$ incurs log-loss $-\log q_i$ when outcome $i$ occurs, the negative of the logarithmic scoring rule. Under $P$, its expected value is the cross-entropy $H(P,Q)=-{\sum }_i{p}_i\log q_i$. Predicting with $P$ itself incurs information entropy $H(P)=-{\sum }_i{p}_i\log p_i$, soThus Kullback-Leibler divergence is exactly excess expected log-loss, whenever these separate quantities are finite. ^[ref]

For lossless source coding, ideal codeword lengths $-{\log }_2{q}_i$ similarly give excess expected length $D_{\mathrm{KL}}(P\Vert Q)$ in bits per symbol. Actual prefix codes require integer codeword lengths; Shannon's source coding theorem explains how long blocks approach ideal per-symbol lengths. The interpretation concerns an average, not a fractional physical bit attached to every outcome. In our example, ideal coding with $Q$ costs one bit per symbol, while $H(P)\approx 0.811278$ bits; the difference is exactly the computed $0.188722$ bits. ^[ref]

Zeros require explicit conventions. An outcome with $p_i=0$ contributes zero, including when $q_i=0$; the convention $0\log 0=0$ agrees with the relevant limit. If $p_i>0$ but $q_i=0$, the Kullback-Leibler divergence is infinite: the predictor has declared impossible an outcome that can occur. On a finite alphabet, the probability support of $P$ must therefore be contained in the probability support of $Q$ for a finite answer. For example, $P=(1,0)$ and $Q=(0.5,0.5)$ give $D(P\Vert Q)=\log 2$, but $D(Q\Vert P)=+\infty$. For general probability measures, failure of absolute continuity of measures $P\ll Q$ likewise gives infinity. Even $P\ll Q$ does not guarantee a finite integral. ^[ref]

To see how confidently wrong predictions are punished, hold the true Bernoulli distribution parameter at $p=0.75$ and vary the predicted parameter $q$. Its binary relative entropy isIt vanishes at $q=0.75$, equals approximately $2.894055$ nats at $q=0.01$, and equals $0.596495$ nats at $q=0.99$. Both endpoints give infinity because both outcomes remain possible under $P$. Predicting almost no successes is especially costly because successes actually occur three quarters of the time. The reversed binary relative entropy stays finite at the endpoints: its averaging probability distribution concentrates on just one outcome, and $P$ assigns positive probability to both.

For model selection, minimizing expected log-loss ranks candidate statistical models by $D(P\Vert Q)$, since $H(P)$ is common to them. The true probability distribution is usually unknown, so one estimates predictive performance using fresh observations or cross-validation. Fitting and assessing on the same data can reward overfitting; a small fitted Kullback-Leibler divergence alone does not establish useful prediction.

In variational inference, a tractable trial probability distribution $R$ commonly minimizes $D(R\Vert \Pi )$ against a target posterior distribution $\Pi$. Maximizing the evidence lower bound accomplishes this without evaluating the target's normalization constant. The averaging now follows $R$: placing trial probability where the posterior distribution assigns almost none is strongly penalized. The chosen direction and trial family therefore matter. ^[ref]

These comparisons do not by themselves establish causal inference. Two statistical models can reproduce the same observed probability distribution while predicting different responses to an intervention. Choosing the smaller Kullback-Leibler divergence therefore needs additional assumptions or experimental evidence before supporting a causal claim. ^[ref]

The original reference is S. Kullback and R. A. Leibler's 1951 paper ^[ref], published in The Annals of Mathematical Statistics 22(1), pages 79–86. Its title connects this comparison with sufficient statistics: the historical setting was statistical information, before today's widespread applications in prediction and variational inference.

The Lasso regularization path follows its coefficients as the penalty or constraint radius changes. Under the usual least-angle construction, the coefficient path is piecewise linear between knots, and variables can enter or leave the active set. Reading support at a selected cross-validation fraction requires checking which coefficients are actually zero at that fraction, rather than retaining only visibly large traces.

Simultaneously applying a permutation to the mixture weights and mixture components leaves $f$ unchanged. Thus identifiability of named components requires a convention, such as ordering distinct component means; label switching describes the associated inferential issue.^[ref]

Even after choosing label names, identical mixture components reveal only their combined mixture weight, and an inactive mixture component has undetermined statistical parameters. Adding mixture components increases flexibility but can cause overfitting; judge predictions using cross-validation, rather than interpreting every fitted component as a discovered population.^[ref]

Nested cross-validation separates selection from assessment using two levels of held-out data. Within each outer training fold, inner cross-validation chooses a model or tuning parameters; the complete selected fitting procedure is then applied to that outer training fold and scored on its untouched outer evaluation fold. This estimates the performance of the model selection procedure, including its choices, rather than reusing its best inner score as an assessment. A final model can then be selected and fitted using all available training data. ^[ref]

Testing the number of components in a finite mixture model is nonregular: absent components have zero weights on the parameter boundary and their means or other parameters are unidentified. The usual chi-squared limit for a likelihood-ratio test therefore need not apply. Information criteria are useful working comparisons, while parametric bootstrap and predictive cross-validation can provide more appropriate evidence. Neither a mixture fit nor a selected component count proves genuine latent populations.

Overfitting occurs when further adaptation improves training error while degrading performance on new data, for example by fitting incidental disturbances instead of a stable regression function. A gap between training error and test error alone does not prove that a simpler rule would generalize better. Compare candidates using cross-validation or a validation set; reserve the test set for assessment. Regularization and early stopping can limit adaptation. ^[ref]

Evaluate the training error example at $z_j=-1+2(j+1/2)/400$, $j=0,\dots ,399$, none coinciding with a training input. The mean squared discrepancy $400^{-1}{\sum }_j(\hat{f}(z_j)-f(z_j){)}^2$ is $0.373259$, $0.000255$, and $0.006319$ for degrees 1, 2, and 7. Thus degree 7 improves training error but worsens recovery of the target relative to degree 2. Degree 1 misses the target's curvature, illustrating underfitting.

These are errors against a known noiseless target, not test errors against fresh noisy responses. In a separate noiseless control, degrees 2 and 7 both recover this target exactly; their numerical held-out errors are approximately $3.77\times 10^{-31}$ and $3.72\times 10^{-31}$. The fixed-disturbance simulation illustrates a mechanism, not a universal numerical benchmark or a claim that higher polynomial degree always harms generalization.

The output must first identify the response scale and the formula. If the initial fit is ordinary least squares to the untransformed response with all fifteen predictors and an intercept, its normal linear model isFor full rank $p=16$, the residual degrees of freedom are $47-16=31$. The residual standard error is $s=\sqrt{\mathrm{RSS}/31}$ and estimated coefficient covariance is $s^2(X^{T}X{)}^{-1}$. A coefficient test divides the estimate by its standard error and compares with a $t_{31}$ reference distribution; the overall test of all fifteen slopes compareswith $F_{15,31}$. The intercept-only residual sum is ${\mathrm{RSS}}_0$. Adjusted R-squared penalizes the loss of residual degrees of freedom, unlike ordinary $R^2$.

Each regression coefficient measures a conditional association holding the other columns fixed. The southern-state indicator compares two state categories at equal values of all other predictors. Rescaling changes the units of a coefficient, so one cannot infer effects per original currency unit or per original percentage unit without the scaling information. If the actual formula uses $\log y$, exponentiating a coefficient gives a multiplicative response change rather than an additive change; if both response and a predictor are logged, the slope is an elasticity. These interpretations cannot be interchanged.

The two police-expenditure variables are likely strongly associated and need a multicollinearity check: individual coefficients can be unstable even when their joint contribution predicts well. Their actual correlation should be calculated, not presumed numerically. Similar considerations apply to age-specific unemployment. With only 47 observations and up to 16 mean parameters, examine regression leverage, Cook's distance, residual distribution, variance versus fitted values, nonlinear effects and predictive cross-validation. State-level geographical dependence may also violate the independent-error model.

The observations are aggregate states. An association between a state covariate and crime rate is not an individual causal effect: ecological fallacy, confounding and reverse causation are relevant. For example, greater recorded crime can induce greater police expenditure. The absent coefficient table and model call prevent deciding which predictors were significant or even whether the fitted response was transformed. The preceding degrees of freedom and formula interpretations are conditional on the explicitly stated full linear-model specification.

For a normal linear model with $n$ fixed observations and $p$ mean coefficients, maximizing over the error variance gives ${\hat{\sigma }}^2=\mathrm{RSS}/n$. The maximized negative twice log-likelihood is $n\log (\mathrm{RSS}/n)$ plus a constant common to the candidate models. Counting the variance as another parameter adds the same penalty two to every model, so the stated criterion is equivalent for comparing them:The Akaike information criterion balances fit against parameter count; smaller values are preferred. It does not compare models using different response scales or different sets of observations without further adjustment.

Stepwise selection by the Akaike information criterion compares allowed one-term additions and deletions, takes a change lowering the criterion, and repeats until no allowed change improves it. The search direction, initial fit and lower/upper scopes matter. A displayed no-change row represents the current model. The term degrees of freedom show how many coefficients a change removes, the residual sum reflects its fit, and the AIC column determines the selected move. The search is greedy, not a guarantee of the best subset over the entire model space. The software's scope and direction conventions are documented in stat.ethz.ch/R-manual/R-devel/library/MASS/html/stepAIC.html .

Deleting $q$ coefficients changes the criterion byThus the AIC deletion threshold in a normal linear model isFor a single coefficient, its partial F-test statistic must be below $(n-p)(e^{2/n}-1)$ for AIC to favor deletion. At the full 47-observation, 16-coefficient fit this is approximately $1.348$, corresponding to $|t|<1.161$. Therefore AIC selection is not equivalent to retaining only coefficients with a conventional five-percent p-value.

The final model should be described by its retained terms, signs, conditional effect sizes, uncertainty, residual diagnostics and prediction behavior. A coefficient that survives selection need not identify a causal mechanism, and its ordinary final-model p-value ignores the preceding search. Stability under bootstrap resampling and predictive cross-validation are useful. The absent search trace and final model prevent identifying the actual deletion sequence, final predictors, coefficients, RSS or AIC. The given starting name does not specify any of those facts.

The family-history table contains $462$ men, of whom $160$ have the response present, giving an overall observed fraction $160/462=0.3463$. The unadjusted fractions are $96/192=0.5$ with a positive family history and $64/270=0.2370$ without one. Their crude odds ratio is $(96/96)/(64/206)=3.219$; this is not the covariate-adjusted effect from the fitted logistic regression.

The full model has an intercept and nine slopes, hence $462-10=452$ residual degrees of freedom. The binomial distribution dispersion is fixed at one. Although the software labels coefficient-to-standard-error ratios as t values, here they are asymptotic normal Wald statistics, not statistics with an estimated Gaussian residual variance and an exact Student reference. The full conditional coefficients provide evidence for tobacco, cholesterol, family history, behaviour score and age: their absolute Wald ratios are approximately $2.99,2.92,4.06,3.22,3.73$. The other four ratios are smaller in magnitude, especially alcohol at about $0.027$. These are conditional associations; a weak adjusted coefficient does not by itself establish absence of an underlying association, and multicollinearity can affect the estimates.

The null deviance is $596.1084$ on $461$ degrees of freedom and the full residual deviance is $472.14$ on $452$. Their difference $123.9684$ has a nominal nine-degree-of-freedom likelihood-ratio test reference for the nine slopes jointly. The absolute residual deviance should not automatically be compared with ${\chi }_{452}^2$ for the reasons in part (i).

The shown stepwise selection by the Akaike information criterion follows a backward path. For individual binary observations the saturated log-likelihood is zero, sowhere $k$ includes the intercept. Removing a one-parameter term improves AIC exactly when the increased deviance is less than two. The successive deletions are alcohol, adiposity, blood pressure and obesity. The corresponding $(D,k,\mathrm{AIC})$ values areAt the final step, every remaining single-term deletion increases AIC. This is a local stopping condition of the search, not a proof that it found the best subset among all possible models. Obesity is retained for several steps because its successive deletion costs differ after the other terms are removed; the procedure does not simply sort the initial Wald ratios.

For the selected fit, write $H=1$ for positive family history and $H=0$ otherwise. The fitted linear predictor isHolding the other predictors fixed, a unit increase multiplies the odds by $e^{{\hat{\beta }}_j}$. In the same variable order, the unit odds ratios areA ten-year age difference multiplies the fitted odds by $1.656$, and ten behaviour-score points by $1.449$. Family history has adjusted odds ratio about $2.48$, with ordinary fitted-model $95\%$ confidence interval $\exp (0.908171\pm 1.96\times 0.225603)\simeq [1.59,3.86]$. The intercept is the log odds at zero numerical covariates and absent family history; that extrapolated baseline is not a typical man's risk. Odds ratios are not risk ratios.

The selected model's residual degrees of freedom are $456$, consistent with its six coefficients. All five retained slopes have nominal normal-Wald $p$ values below $0.004$ in that model, but these tests and confidence intervals do not incorporate the prior model selection. Its improvement over the intercept-only fit is $596.1084-475.6856=120.4228$ on five nominal degrees of freedom; it loses only $3.5456$ deviance units relative to the nine-slope fit while using four fewer parameters. Selection optimism remains relevant to predictive performance. Check possible nonlinear continuous-predictor effects, influential observations and clinically motivated interactions; assess out-of-sample discrimination and agreement of predicted probabilities with observed frequencies by cross-validation or external data. The printed output alone does not supply those diagnostics, and the selected high-risk male population limits extrapolation to other populations. Nothing here establishes causal effects of these observational predictors.

The response is the species categorical variable, while the four measurements are candidate predictors. The dot in the formula means all other columns of the data frame. The fitted object is a classification tree: it recursively partitions predictor space by binary threshold rules and assigns a species and class-probability vector to each terminal region. Sepal width was available but was not selected in the displayed tree.

At a node $t$, let $n_{tk}$ be the number of observations of class $k$ and $n_t={\sum }_k{n}_{tk}$. The multinomial likelihood is maximized by ${\hat{p}}_{tk}=n_{tk}/n_t$. The fitted class is one attaining the largest proportion, with a software convention resolving ties. The classification-tree deviance isA pure node has zero deviance. At each nonterminal node, consider candidate split points between distinct observed predictor values, and choose the split with the largest decrease $D(t)-D(t_L)-D(t_R)$. Repeating this local maximization constructs a binary decision tree; it does not solve a global optimization over all trees. Growth stops when nodes are pure or size/deviance controls prohibit further splitting. Mixed leaves with five or six observations can therefore remain. The node number $j$ has children $2j$ and $2j+1$, and a terminal-node marker means that no further split was retained.

The probability triples are in class order $(c,s,v)$, not $(s,c,v)$. For example, the pure left leaf has probability vector $(0,1,0)$ and predicts $s$, while the 54-observation node has counts $(49,0,5)$ and predicts $c$. The root counts are $(50,50,50)$, giving $D=300\log 3\simeq 329.584$ and a tied majority class. Its reported choice of $c$ does not indicate a more common class.

The complete derived decision rule is shown below. Each leaf includes its fitted class and the observed class counts, so both the branching logic and its remaining errors are visible. For the observed data, the printed strict inequalities leave no equality cases because the thresholds fall between measurement values. A consistent right-branch convention at equality extends the rule to new specimens.

The six terminal nodes contain the class countsThus the training misclassification error isThe terminal classification-tree deviance is insteadThe displayed residual mean deviance divides this by $150-6=144$, yielding approximately $0.1253$. This is the summary's reporting convention, not a normal-error residual mean square or an automatic chi-squared calibration with 144 statistical degrees of freedom. The two summary measures answer different questions: deviance depends on the fitted probability of each observed class, while misclassification counts only whether the majority-class prediction is correct. Both are training quantities. Cross-validation or independent test data, rather than the displayed training error, are needed to assess prediction on new specimens; cost-complexity tree pruning can select a less elaborate tree if predictive performance warrants it.

princomp(): principal component analysis. Supply observations in rows and numeric variables in columns. After centering, let $Z$ denote the resulting matrix and $C=Z^{T}Z/n$ its sample covariance matrix, with an explicitly stated normalization. WriteThe first direction maximizes the Rayleigh quotient $q^{T}Cq$ over unit vectors; successive directions maximize it subject to orthogonality to the earlier directions. The principal component scores are $z_{ij}=q_j^T(x_i-\bar{x})$, with variances ${\lambda }_j$ and zero pairwise sample covariance. Zero covariance alone does not imply independence. Each principal component loading describes a direction in variable space; its sign is arbitrary, and repeated eigenvalues leave the basis inside the corresponding eigenspace arbitrary.

The function returns principal component loadings, component standard deviations sdev and, when requested, principal component scores. By default it uses a covariance matrix; cor=T instead performs principal component analysis on a correlation matrix, standardizing nonconstant variables to variance one. This choice matters when variables have different measurement units. These interface details are documented in the S-Plus help for princomp.

A scree plot shows the ordered eigenvalues; a plot of the first two principal component scores can reveal groups, unusual observations or a dominant gradient. The first $k$ components explain ${\sum }_{j\le k}{\lambda }_j/{\sum }_j{\lambda }_j$ of the total variance. Reconstructing each centered observation by its orthogonal projection onto these directions minimizes average squared reconstruction error among $k$-dimensional linear subspaces. Thus PCA gives an optimal linear compression for squared reconstruction error, with scaling and the retained dimension chosen explicitly. It is an unsupervised learning method, rather than a classification rule fitted to supplied labels.

tree(): classification trees and regression trees. A call such as tree(response ~ predictors, data=dat) fits a recursive partitioning model. At a current node, test numerical predictors against candidate thresholds, or divide the levels of a categorical predictor into two sets. Select a split that most improves the fitting criterion, then repeat within its children. Numerical splits give axis-aligned regions. A categorical response produces a classification tree; a numeric response produces a regression tree, as specified in the S-Plus tree documentation.

For a regression tree, the constant prediction in a terminal region is its sample mean, minimizing the sum of squared residuals there. For a classification tree, estimate class probabilities by the observed proportions in a leaf and predict a most probable class under equal misclassification costs. The classification-tree deviance of a node iswith $0\log 0=0$; splitting maximizes the decrease from parent to children. This fitting criterion differs from the number of wrong classifications. The middle sketches show the same three-region partition as a binary tree: first split on variable one, then split the right-hand branch on variable two.

Growing to small leaves can overfit. Use cost-complexity tree pruning, balancing fitted loss $R(T)$ against leaf count,and select complexity by cross-validation. Plotting the fitted tree and annotating nodes displays the rules; prediction routes each new observation through the fitted splits. A tree gives an interpretable, piecewise-constant prediction rule; training fit alone does not measure prediction accuracy.

The rpart() alternative. This also fits classification and regression trees by recursive partitioning, with squared-error splitting for ordinary numeric responses and, by default for classification, Gini impurity $1-{\sum }_r{p}_r^2$. Classification settings can instead use information-based splitting and specify prior probabilities and a loss matrix. It supports surrogate splits to approximate a node's primary left/right assignment when the primary predictor is missing. Its complexity-parameter path and cross-validated errors support cost-complexity tree pruning; for example, select a complexity parameter and apply prune(fit, cp=...). The maintainer's rpart documentation describes the criteria and the distinction in handling surrogate predictors. The partition and tree sketches apply to either implementation; the fitted tree need not be the same because splitting criteria and controls can differ.

cmdscale(): classical multidimensional scaling. Its input is a dissimilarity matrix or stored distance object. It outputs coordinates for the observations in a chosen dimension, usually two for visualization. Let $D^{(2)}$ contain the squared entries of $D$, and putFor centered Euclidean coordinates $X$, the distance identity $d_{ij}^2=\Vert x_i{\Vert }^2+\Vert x_j{\Vert }^2-2x_i^T{x}_j$ gives $G=X{X}^T$: multiplying by $H$ removes the row and column norm terms. Diagonalize this Gram matrix and retain its $k$ largest positive eigenvalues. The coordinates areThus classical scaling exactly recovers Euclidean distances when all positive eigenvalues are retained. Reducing dimension approximates the centered Gram matrix; it does not in general minimize squared discrepancies in the original distances. Negative eigenvalues identify a failure of the Euclidean distance matrix criterion, so their size matters when interpreting a display. The historical S-Plus cmdscale help describes the dimension argument, eigenvalue output and optional additive adjustment to dissimilarities.

Plot the two coordinate columns on equally scaled axes and label observations. Nearby plotted points suggest small dissimilarities only insofar as the dimension reduction fits the input well. The bottom sketches give an exact Euclidean example: a configuration and its independently recovered two-dimensional coordinates have identical pairwise distances, despite different coordinate axes. Classical scaling converts distances into geometry; translations, rotations and reflections do not change the represented distances. For Euclidean distances between observations, its coordinates agree with the corresponding principal component scores up to these harmless choices of axes.

The formula makes species the categorical response and the dot denotes all four other columns as candidate predictors. The classification tree is grown by recursive binary partitioning, not by fitting a linear regression to numerical species codes. At a node $t$, count observations of each class, $n_{tk}$, and set ${\hat{p}}_{tk}=n_{tk}/n_t$. These are the multinomial maximum-likelihood probabilities. The classification-tree deviance isFor each candidate predictor and threshold, calculate $D(t)-D(t_L)-D(t_R)$ and choose the largest permitted decrease. Continue recursively until a node is pure or the stopping controls prevent another split, for example because it has too few observations or insufficient deviance to warrant further growth. Only petal length, petal width and sepal length appear in the selected splits; sepal width was available but did not win a split. This greedy construction need not find a globally optimal tree. The default deviance criterion and recursive partitioning are documented in the tree package manual.

Each node number encodes its position: node $j$ has left and right children $2j$ and $2j+1$. The reported sample size is the number reaching the node; its deviance is calculated from that node's class counts. The fitted class is the class with largest fitted probability, with a convention resolving ties. The root's three counts are equal, so its displayed class is only a tie choice. An asterisk marks a terminal node, where prediction stops.

The resulting rule is displayed below. A left edge satisfies the stated strict inequality and a right edge its complement. None of the recorded training values lies exactly at these midpoint thresholds; the diagram adopts the right branch at equality for prediction.

Its six terminal regions have the following derived class counts, in the order setosa, versicolor, virginica:The root split identifies all 50 setosa observations. Among the remaining specimens, the petal-width split is the principal separation of versicolor from virginica; further petal-length and sepal-length splits refine it. The leaf errors add to $4/150\simeq 2.67\%$. The leaf deviances add toDividing by the software's residual degrees-of-freedom convention $150-6=144$ gives $18.0489/144\simeq 0.1253$, as reported. The root deviance is $-2(150)\log (1/3)=300\log 3\simeq 329.6$. The deviance is a likelihood measure, whereas the error rate counts wrong majority-class predictions; these are different fit summaries.

Both quantities are in-sample, so the low error does not establish prediction accuracy on new specimens. Cost-complexity tree pruning, with tree size chosen by cross-validation, can remove weak splits and reduce overfitting. An independent test set is needed for a final estimate of the selected tree's prediction error.

Fit the finite Gaussian mixture with a common variance for candidate values $k=1,\dots ,K$, using several initializations of the expectation-maximization algorithm for each, and retain the best likelihood value found. Maximized likelihood alone cannot select the number of components: a smaller mixture embeds in a larger one by giving an added component zero weight, so its likelihood cannot decrease.

There are $k-1$ free mixture weights, $k$ means and one common variance, henceA standard working choice is the Bayesian information criterion:The penalty balances fit against parameter count. The Akaike information criterion instead uses $-2\ell ({\hat{\theta }}_k)+4k$ and is another possible comparison, particularly when predictive fit is the goal. The parameter count would differ if component variances were separate, but that is not this model.

Likelihood fitting must first be made well posed. Even a shared variance does not prevent all degeneracy: if $k=N$, choose ${\mu }_j=x_j$, ${\alpha }_j=1/N$ and let the common variance tend to zero. Each observed density is at least $1/(N\sqrt{2\pi {\sigma }^2})$, so the full log-likelihood function tends to $+\infty$. The same construction works when $k$ reaches the number of distinct observed values. Thus one must not feed infinite fitted likelihoods into an information criterion. Restrict candidate sizes to a sensible range below that degeneracy, or impose a fixed positive variance floor and use that same constrained model in every comparison. Under a variance floor $\epsilon$, the M-step variance becomes $\max \{\epsilon ,R/N\}$.

There is also nonregular mixture model selection: at a missing component its weight is on a boundary and its mean is unidentified, while permuting labels leaves the distribution unchanged. Consequently an ordinary chi-squared calibration for a likelihood-ratio test between successive component counts is not generally justified. Information criteria are practical comparisons, not an automatic universal proof that the true number has been recovered. Predictive cross-validation or a suitably constructed parametric bootstrap comparison can supplement them. Select the component count by a penalized, well-defined mixture fit, rather than by unpenalized likelihood increase.

One useful starting point is reverse correlation, implemented as a spike-triggered average. Present a long stimulus record and retain the stimulus history before each observed spike. After subtracting the stimulus mean, estimatewhere $K$ is the number of spikes and $T_k$ is the history window. This identifies a candidate temporal receptive field. For white Gaussian stimulation and a one-filter encoding model, the average is proportional to the relevant filter, provided the response nonlinearity does not make that average vanish. For example an even response to the projection can have zero average despite strong stimulus dependence. Colored inputs require accounting for stimulus covariance, and more than one relevant stimulus direction may require a richer fitted model.

Use the candidate filter in an LNP model:Estimate the static nonlinearity $g$ by binning filter projections and dividing spike counts by exposure time in each bin. Equivalently, Bayes' rule gives $g(z)=\overline{r}p(z\mid \text{spike})/p(z)$ under this model. Refine the filter and nonlinearity by maximizing the conditional Poisson log-likelihoodwith suitable regularization. Simulate an Inhomogeneous Poisson process at the fitted rate using integrated-intensity inversion from the previous solution. This gives a statistical encoding model, not a claim to reconstruct the neuron's ion channels.

Use cross-validation: estimate parameters on one set of trials and test them on independent held-out trials. Under repeated identical stimuli, compare the observed and simulated trial-averaged firing rates, spike-count distributions, interspike-interval distributions and autocorrelations. Compare held-out likelihoods and predictive errors with simpler baseline models and with the variability between real repeats. Exact spike-by-spike agreement is not expected from two independent realizations of a stochastic model. Large rate agreement accompanied by incorrect short-interval or correlation statistics reveals missing dynamics. Adding a spike-history term, for example in a point-process generalized linear model, can account for refractoriness or bursting when the basic Poisson cascade fails.

A good model predicts held-out response distributions, not just its training spike train. Such a model summarizes what stimulus features affect the neuron, permits controlled simulated changes of those features, and tests whether the proposed computation is sufficient. Its failures can suggest further experiments or additional stimulus and history variables.

Computational models and biological experiments answer complementary questions. Experiments constrain what a nervous system actually does; a model makes a proposed explanation precise enough to test whether its mechanisms can produce those observations. The useful comparison is between competing, experimentally constrained explanations, rather than between modelling and experimentation as alternatives. A good model exposes assumptions, predicts new observations and identifies measurements that would distinguish it from rivals.

A Hodgkin-Huxley model links action potentials to voltage-dependent conductances and channel kinetics. It is more informative than merely fitting a spike waveform because changes in its conductances predict changes in excitability. Yet a model calibrated for one cell or preparation does not automatically represent another: different channel combinations and spatial geometry can yield similar outputs. Detailed parameters must be constrained by experiments, and apparent agreement does not make them uniquely identifiable. A leaky integrate-and-fire model deliberately sacrifices channel detail to study the effects of connectivity, timing and synapses in a large network. It is useful for such questions even though it cannot explain the microscopic origin of the action-potential waveform.

At a functional level, an LNP model tests which stimulus features predict a neuron's response and how those features map to firing rate. Independent testing, including response correlations rather than only mean rate, reveals what its assumptions miss. It can characterize a receptive field without claiming that the brain explicitly performs its particular fitted calculation. A Hopfield network likewise demonstrates that recurrent interactions can implement content-addressable memory: corrupted cues relax toward stored patterns. Its usefulness is an existence proof and a quantitative account of interference, not evidence that biological memory literally uses symmetric binary connections and the same update schedule.

Developmental models connect observed input statistics with synaptic plasticity. Hebbian learning with normalization or synaptic competition can produce selective weights and segregation of competing inputs. Models of correlated ON and OFF inputs can explain the formation of oriented simple-cell subregions and predict how changing the correlations changes their organization. That makes altered visual experience or patterned stimulation informative tests. Such models still need anatomical, developmental and plasticity constraints; several learning rules may reproduce similar final receptive fields. A primary model of activity-dependent ON/OFF competition illustrates this approach.

Normative models instead ask what representation would perform a specified task efficiently. For example, sparse coding of natural images can produce localized oriented filters resembling simple cell receptive fields. This links environmental statistics to a useful coding objective, but resemblance alone neither proves that the objective is the only explanation nor specifies the biological learning mechanism. Predictions about altered stimulus ensembles, population activity and synaptic changes provide stronger tests. The original sparse-code receptive-field model provides an example.

These distinctions also explain the dangers of models. Excessive flexibility can fit noise; unrealistic assumptions can generate a desired effect by construction; and multiple parameter sets can have nearly indistinguishable outputs. Simulation without analysis may obscure rather than explain a mechanism. Cross-validation, uncertainty analysis, comparisons with simpler models and controlled perturbations are therefore important. Conversely, collecting more measurements without a discriminating hypothesis can leave the same explanatory ambiguity unresolved.

The productive cycle is to propose a constrained model, determine which observations it predicts beyond the fitting data, perform the relevant experiment, and revise or reject the model. Models help select experiments and interpret results; experiments establish their scope and reveal omitted mechanisms. The appropriate level of detail depends on the question: spike-generation mechanisms, circuit computation and developmental organization need not be explained by the same model.

Fit both hierarchies to the historical data and compare their posterior predictive checks. Use discrepancies sensitive to between-individual tails, such as the maximum standardized personal mean, tail counts and the spread of individual sample means. Simulate new individual means as well as repeated observations; conditioning only on fitted personal means would conceal a poor population model.

Leave whole individuals out for cross-validation, rather than just measurements, when the aim is prediction for a new individual. Compare held-out predictive log densities or interval calibration, with uncertainty in the comparison. Check both fits' Markov chain Monte Carlo convergence diagnostics and sensitivity to scale priors. Better tail prediction and adequate calibration would support the heavier-tailed model; the data are required to decide, so no empirical winner can be asserted here.