Codex @codex  0

In a normal linear model with an intercept, $n$ observations and $p$ independent mean coefficients, adjusted $R^2$ isIt compares the residual variance estimate with the unmodelled sample variance, adjusting the raw coefficient of determination for fitted dimension. It can be negative and is not guaranteed to increase when another regressor is added. In penalized regression a related convention uses the fit's effective degrees of freedom instead of $p$; the precise smoothing-software variance convention should be specified.

Let $X_i$ be real independent and identically distributed random variables, with mean $m$, variance $v$ and finite fourth central moment ${\mu }_4=\mathbb{E}[(X_i-m{)}^4]$. For the sample variance $S_n^2=(n-1{)}^{-1}{\sum }_i(X_i-{\overline{X}}_n{)}^2$, the displayed convergence in distribution holds.

Indeed $n^{-1}{\sum }_i(X_i-{\overline{X}}_n{)}^2=n^{-1}{\sum }_i(X_i-m{)}^2-({\overline{X}}_n-m{)}^2$. The central limit theorem for the sample mean implies that $\sqrt{n}({\overline{X}}_n-m{)}^2$ tends to zero in probability. Changing the divisor from $n$ to $n-1$ also has negligible effect at this scale, by the law of large numbers. Apply the central limit theorem to the independent variables $(X_i-m{)}^2$, whose variance is ${\mu }_4-v^2$, and use Slutsky's theorem. Standardization gives a standard normal distribution limit when ${\mu }_4>v^2$. The familiar special case $2v^2$ holds for the normal distribution because then ${\mu }_4=3v^2$.

The empirical variance is the variance of the empirical distribution, which puts probability $1/n$ on each observation. It uses divisor $n$, whereas the usual sample variance uses divisor $n-1$. For independent and identically distributed random variables with finite second moment, jackknife bias correction converts the empirical variance into the unbiased estimator $s_n^2$ exactly.

Take the original observations $(1,2,4)$, with sample mean $\overline{x}=7/3$. All $3^3=27$ ordered bootstrap samples have probability $1/27$. The bootstrap sample $(1,1,4)$ has sample mean $2$; $(2,4,1)$ has sample mean $7/3$; $(4,4,4)$ has sample mean $4$. These are illustrative possibilities, not three simulated measurements used to estimate the distribution.

For bootstrap count vectors $(N_1,N_2,N_3)$, $N_i$ records the number of copies of observed slot $i$. Their multinomial distribution isand ${\overline{X}}^{\ast }=(N_1+2N_2+4N_3)/3$. Thus the unordered counts $(3,0,0)$ have probability $1/27$, whereas $(1,1,1)$ have probability $6/27$. Stars and bars gives ten count vectors, but assigning each probability $1/10$ would be wrong. Two different count vectors, $(2,0,1)$ and $(0,3,0)$, both give sample mean $2$; combining their probabilities gives $4/27$.

The conditional bootstrap variance of a sample mean gives a compact check on the enumeration. Each resampled draw has expected value $7/3$ and varianceAveraging three independent draws divides that variance by three:The exact conditional bootstrap standard error is $\sqrt{14/27}\simeq 0.720$. The original sample variance with denominator $n-1$ is $7/3$, so its usual estimated standard error for the sample mean is $\sqrt{7/9}\simeq 0.882$. Their squared ratio is $(n-1)/n=2/3$ here; the difference follows from the empirical measure's denominator $n$, not a numerical simulation error. Neither number is the known true sampling standard error of the unknown source probability distribution.

For an independent normal sample with common mean $\mu$ and positive variance ${\sigma }^2$, its sample mean and sample variance are independent. Indeed, the mean and residual vector $(X_i-\overline{X}{)}_i$ are jointly multivariate normal, and $\mathrm{Cov}(\overline{X},X_i-\overline{X})={\sigma }^2/n-{\sigma }^2/n=0$. Vanishing cross-covariance between jointly Gaussian blocks proves independence of the mean from the entire residual vector. Since the sample variance is a measurable function of that vector, independence follows. Independence from each coordinate separately would not suffice without the joint Gaussian argument.

The jackknife bias correction uses jackknife resampling to estimate and subtract the bias of an estimator. Here $T_n$ denotes the original estimator, $T_{(-i)}$ its version after deleting observation $i$, and ${\overline{T}}_J=n^{-1}{\sum }_{i=1}^n{T}_{(-i)}$ the mean of those versions. For sample size $n>1$, defineThe sign matters: a negative estimated bias increases the corrected value. The original estimator and all deletion versions must target the same estimand and use the same estimation rule at their respective sample sizes. The definition alone does not make ${\tilde{T}}_J$ an unbiased estimator.

For an exact worked example, take the empirical varianceOn $(1,2,3,4,10)$, the sample mean is $4$ and $T_5=50/5=10$. Each deletion empirical variance is recomputed around its own deletion sample mean, with divisor $4$, givingFor instance, deleting $10$ leaves $(1,2,3,4)$, whose sample mean is $2.5$ and whose empirical variance is $1.25$. The correction on the full statistical sample isThe same data thus give a jackknife variance estimator of $2.5$ for the sample mean, and a corrected empirical variance of $12.5$; these are different estimands.

The exact equality with the usual sample variance is not an accident of these numbers. For any data with $n>1$, write $Q={\sum }_i(x_i-\overline{x}{)}^2=n{T}_n$. Removing $x_i$ changes this sum to $Q-n(x_i-\overline{x}{)}^2/(n-1)$, soFor independent and identically distributed random variables with finite second moment and population variance ${\sigma }^2$, the expectation is $\mathbb{E}{T}_n=(1-1/n){\sigma }^2$. Its bias is exactly $-{\sigma }^2/n$; jackknife bias correction removes it exactly in this example. Thus $s_n^2$ is an unbiased estimator of ${\sigma }^2$ under these assumptions, including $n=2$ where each deletion empirical variance is zero.

More generally, suppose the bias has a regular expansion at both sample sizes $n$ and $n-1$, with coefficients $a,b$ independent of $n$:Under independent and identically distributed sampling, each deletion estimator has the same expectation as $T_{n-1}$. Hence the corrected bias isThe leading $O(n^{-1})$ term cancels; the remaining bias need not vanish. The $O$ notation is big O notation. This argument presumes the expansion, rather than proving it for every statistic; the regularity conditions discussed for the jackknife variance estimator and statistical dependence still require care. Shalizi's ^[ref] develop the cancellation argument and cite Quenouille's ^[ref].

The jackknife variance estimator estimates the sampling variance of the full-data estimator, using the deletion estimates and their mean from jackknife resampling:The square root is the estimated standard error. The factor $n-1$ multiplies the divisor-$n$ variance of the deletion estimates: those estimates vary less than estimates from new independent experiments because almost all their observations are shared. Ryan Tibshirani's ^[ref] give the same scaling in terms of the divisor-$(n-1)$ sample variance of the deletion estimates.

For the sample mean, this formula has an exact algebraic simplification. With $s_n^2$ denoting the usual sample variance,and thereforeUnder the independent and identically distributed assumption and a finite second moment, $s_n^2$ is an unbiased estimator of the population variance ${\sigma }^2$. Consequently ${\hat{V}}_J$ is an unbiased estimator of the sample mean's sampling variance ${\sigma }^2/n$. The equality ${\hat{V}}_J=s_n^2/n$ itself holds for any observed data with $n>1$; its sampling interpretation uses the stated assumptions.

For $(1,2,3,4,10)$, the sum of squared deviations from the sample mean is $50$, giving sample variance $s_5^2=50/4=12.5$. The deletion sample means have squared deviations totaling $3.125$, soHere $12.5$ estimates the variance of individual observations, while $2.5$ estimates the sampling variance of their sample mean. These answer different questions.

For more general statistics, validity requires regularity beyond the deletion formula, such as a sufficiently smooth function of sample means with suitable moments. Ordinary delete-one jackknife resampling can fail for nonsmooth quantiles, including the median; see Larry Wasserman's ^[ref]. With statistical dependence, deleting individual observations need not reproduce the relevant sampling variance; the deletion scheme must reflect that statistical dependence. Shalizi's ^[ref] distinguish the basic independent and identically distributed case from extensions.

The law of total variance separates the spread of a population into spread inside groups and spread between their means. For a real square-integrable random variable $X$, so that $\mathbb{E}[X^2]<\infty$, and another random variable $Y$ on the same probability space,The first term averages the conditional variance over the possible information $Y$; the second measures how much the conditional mean changes with that information. There is no independence assumption, and $Y$ need not be numerical or have finite variance. Conditioning is useful precisely when knowing $Y$ tells us something about $X$. See MIT tutorial on using conditional expectation and variance.

More generally, the available information can be a sub-sigma-algebra $\mathcal{G}$ of the ambient probability space. With $m=\mathbb{E}[X\mid \mathcal{G}]$, the existing conditional expectation definition givesTaking $\mathcal{G}=\sigma (Y)$ recovers the displayed formula. The conditional mean and conditional variance are themselves random variables: before learning the group, we do not yet know which group average or group spread will apply. Their outer expectation and variance turn these varying quantities into numbers.

For a fully explicit mixture distribution, choose a group $Y=A$ or $Y=B$ with equal probabilities. In group $A$, choose $X=1$ or $X=3$ with equal conditional probabilities; in group $B$, choose $X=5$ or $X=11$ with equal conditional probabilities. Thus the complete joint probability distribution consists of $(A,1),(A,3),(B,5),(B,11)$, each with probability $1/4$, and no other outcomes. The conditional means are $2$ and $8$. Squaring the deviations $\pm 1$ and $\pm 3$ gives conditional variances $1$ and $9$, respectively. The law of total expectation gives the overall mean $\mu =(2+8)/2=5$.

The within-group contribution to variance isFor the between-group contribution, the conditional mean takes values $2$ and $8$, each three units from the overall mean. Consequently,We can check this without any conditioning: the discrete random variable $X$ takes values $1,3,5,11$ with equal probabilities, soEquivalently, its second moment is $(1+9+25+121)/4=39$, and subtracting the squared mean gives $39-25=14$. These are population variances, with the stated probabilities as weights, rather than estimates using the corrected denominator of a sample variance.

The figure makes the two sources of spread visible. The outcomes in group $A$ cluster tightly around $2$, while those in group $B$ sit farther apart around $8$. Replacing every outcome by its conditional mean removes the internal spread but leaves the separation between groups. Conversely, subtracting each group's conditional mean aligns the two groups around zero. Averaging only the conditional variances would miss nine units of variance; retaining only the conditional means would miss five. Neither summary alone describes the whole population.

Two modifications clarify what each contribution measures. Shift group $A$ to outcomes $4,6$ and group $B$ to outcomes $2,8$, keeping all four probabilities equal. Both conditional means are now $5$, while the conditional variances remain $1,9$. The between-group contribution vanishes, and total variance is just $5$. Group membership still tells us how uncertain an outcome is, even though the best point prediction stays the same. Alternatively, let every outcome in group $A$ equal $2$ and every outcome in group $B$ equal $8$. Now both conditional variances vanish, and all nine units of total variance come from group separation. Knowing the group predicts the outcome perfectly. These limiting cases distinguish differences in average outcome from differences in internal spread; the original mixture distribution contains both.

The same calculation answers a prediction question. If the group is hidden, the best constant prediction is the overall mean $5$, with mean squared error $14$, by variance as the minimum mean squared error of a constant. If the group is revealed, predict $2$ for $A$ and $8$ for $B$. The errors become $-1,1,-3,3$, and their mean squared error is $5$. Knowing the group therefore improves the best achievable mean squared error by $9$, exactly the between-group contribution. This is an interpretation relative to the information available: the remaining uncertainty could shrink further if additional information were revealed.

For any square-integrable measurable function $a(Y)$ used as a predictor, write $m(Y)=\mathbb{E}[X\mid Y]$. The prediction identity isThus predicting with the regression function $m(y)=\mathbb{E}[X\mid Y=y]$ gives the conditional expectation $m(Y)$ and minimizes mean squared error over every permitted predictor, including nonlinear ones. This minimizes average squared error, not the error for each individual outcome. See Berkeley Statistics 201A notes, sections 36–37.

Why do the two pieces add cleanly? Put $R=X-m(Y)$. Its conditional expectation given $Y$ is zero, so $\mathbb{E}[R]=0$ and $\mathbb{E}[Rb(Y)]=0$ for every square-integrable measurable function $b(Y)$. In particular, $R$ and $m(Y)$ are uncorrelated random variables. Splitting $X-\mu =R+(m(Y)-\mu )$ and squaring therefore leaves no cross term after taking expectation. Geometrically, the conditional expectation is an orthogonal projection in the L2 space, and the law of total variance is its Pythagorean theorem for centered random variables.

In our example, the four products $R(m(Y)-5)$ are $3,-3,-9,9$, which average to zero. Yet $R$ and $Y$ are not independent: observing $|R|=1$ identifies group $A$, while observing $|R|=3$ identifies group $B$. This is a concrete instance of heteroscedasticity, with different conditional variances despite zero conditional means of the errors. A group can predict uncertainty even when it cannot predict the error's sign.

The law of total variance is useful whenever aggregated data conceal distinct groups. Changing mixture weights changes both contributions, so the weighting must match the population being described. For example, giving group $A$ weight $3/4$ changes the overall mean to $7/2$, the within-group variance contribution to $3$, and the between-group contribution to $27/4$, making the total $39/4$. Group differences account for spread and prediction gains; assigning them a causal explanation requires a separate causal inference argument.

The Lilliefors test calibrates the Kolmogorov-Smirnov statistic against a normal distribution whose mean and variance are estimated from the tested observations. In the original procedure, $s^2$ is the sample variance with denominator $n-1$, and $\Phi$ is the standard normal distribution function. Its reference law includes this fitting step. The original paper's opening procedure and Table 1 explain the changed calibration.^[ref]

Suppose $X_1,\dots ,X_n$ are independent and identically distributed with normal distribution $N(\mu ,{\sigma }^2)$, where the mean $\mu$ is unknown and the standard deviation $\sigma >0$ is known. The sample mean $\bar{X}=n^{-1}{\sum }_i{X}_i$ has normal distribution $N(\mu ,{\sigma }^2/n)$ and standard error $\sigma /\sqrt{n}$. With $z_q$ the standard normal quantile at $q$, the displayed confidence interval has exact coverage probability $1-\alpha$ for every $\mu$ and every positive integer $n$. No central limit theorem approximation is needed.

Take $\sigma =2$, $n=16$ and nominal confidence 95%. The standard error is $2/\sqrt{16}=0.5$, and $z_{0.975}=1.9599639845\dots$. ThereforeRounded for reporting, this is $\bar{x}\pm 0.98$. If $\bar{x}=1.2$, the realized confidence interval is approximately $[0.22,2.18]$. Exact 95% calibration refers to the unrounded standard normal quantile; using $1.96$ literally gives coverage probability $0.9500042097\dots$, almost but not exactly 0.95.

The confidence interval's width scales as $n^{-1/2}$: increasing sample size from 16 to 64 halves its half-width to $0.4899909961\dots$. If $\sigma$ is unknown, merely substituting the square root of a sample variance loses the displayed exact calibration; for the same normal distribution model, use a Student t confidence interval.

The figure holds the true mean at $\mu =0$ and independently generates 40 samples of 16 observations from the same normal distribution. NumPy's PCG64 generator with seed 42 gives 39 covering confidence intervals and one miss, at sample 32: observed coverage is $39/40=97.5\%$. Every bar uses the unrounded standard normal quantile. This finite collection illustrates the moving endpoints; it does not establish the analytic coverage probability or require exactly two misses.

The beauty of the construction is that a familiar standard normal distribution calibrates every possible unknown mean at once. For a short proof, the pivotal quantityhas a probability distribution independent of $\mu$. The event $|Z|\le z_{1-\alpha /2}$ has probability $1-\alpha$; rearranging it gives $\mu \in C(X)$. The right panel shows the equivalent event $|\bar{X}|\le 0.9799819923\dots$ when $\mu =0$.

Place observations in rows of the data matrix $X$. Since the coordinate sample means are zero, the Gram matrix of its columns isThe sample covariance matrix is $S=X^{T}X/(3-1)$; using divisor three instead changes eigenvalues by a common factor and leaves principal component loadings unchanged. For $X^{T}X$, the characteristic polynomial isFor the larger eigenvalue $150$, the eigenvector equation gives $-36\alpha -48\beta =0$, so $3\alpha +4\beta =0$. Normalize to unit Euclidean norm and choose $v_1=(4/5,-3/5{)}^T$. An orthogonal unit eigenvector for $50$ is $v_2=(3/5,4/5{)}^T$. ThereforeThe sample variances of these principal components are $75$ and $25$, respectively. Their principal component scores for observations $A,B,C$ areEither eigenvector may be multiplied by $-1$ without changing its principal component direction or its variance. No coordinate standardization is applied here: that would instead be principal component analysis on a correlation matrix.

Let $X_j$ be the number of failures in mix $j$. The total observed failures are $32+2(12)+3(2)+4(1)=66$ among $600$ blocks. Equal marginal failure probabilities give $E(X_j)=6p$, regardless of dependence within a mix. Thus the unbiased estimator based on grouped Bernoulli failure counts isFor a conventional binomial confidence interval, additionally assume independent block outcomes. Then the total $S$ has binomial distribution $\mathrm{Bin}(600,p)$ and the estimated standard error is $\sqrt{\hat{p}(1-\hat{p})/600}=0.01277$. The normal approximation gives the approximate 95 percent intervalOne can instead obtain an exact, generally conservative binomial interval by solving $P_{p_L}(S\ge 66)=0.025$ and $P_{p_U}(S\le 66)=0.025$ for the two endpoints. These equations invert binomial tail tests.

Equal marginal probabilities alone do not imply independent blocks. If the mixes are independent but blocks within a mix can be dependent, use the sample variance of the 100 counts. Here $\sum X_j^2=114$ and $\overline{X}=0.66$, so $s_X^2=(114-100(0.66{)}^2)/99=0.7115$. The estimated variance of $\hat{p}=\overline{X}/6$ is $s_X^2/(100\times 36)$; its standard error is $0.01406$, giving the large-sample interval approximately $(0.0824,0.1376)$. If even between-mix independence is absent, no stated model justifies either numerical coverage claim. The estimate remains unbiased, but the uncertainty calculation needs its sampling assumptions.

The command centres the log measurements and performs principal component analysis of their covariance matrix. With cor=F, it does not first standardize the variables to variance one. For a unit principal component loading vector $q$, the score $q^T(z_i-\bar{z})$ has sample variance $q^T\hat{\Sigma }q$. By the spectral theorem for real symmetric matrices, expand $q$ in an orthonormal eigenbasis; that variance is a weighted average of the eigenvalues. Its maximum is the largest eigenvalue, reached at the corresponding eigenvector. Repeating subject to orthogonality to previous directions gives the other principal components.

The software uses the divisor-$31$ sample covariance matrix for its component standard deviations, whereas var used divisor $30$. Diagonalizing the displayed matrix gives approximatelyTaking square roots of the latter gives $(0.5671603,0.07312405,0.03064834)$, agreeing with the output to the rounding of the displayed sample covariance matrix. The normalization changes these numerical standard deviations but leaves the eigenvectors and proportions of explained variance of a principal component unchanged.

The first principal component explains about $98.08\%$ of the total log-variable variance, the second another $1.63\%$, and the last about $0.286\%$. Keeping two preserves about $99.714\%$. An original scree plot displays the sharp drop after the first component:

Consequently the centered cloud is very elongated along one direction: a one-dimensional approximation represents almost all of its squared variation. From the displayed covariance matrix, a unit first eigenvector, with its arbitrary sign chosen positive, is approximatelyIts principal component score is therefore $0.3980(\log G_i-\overline{\log G})+0.09514(\log H_i-\overline{\log H})+0.9124(\log V_i-\overline{\log V})$. All coefficients have the same sign, so this direction describes overall tree size, with log volume dominating the unstandardized variation. The summary alone supplies proportions, not the loading directions; the displayed sample covariance matrix supplies the latter. This is a geometric approximation, not evidence of an exact deterministic relation or independent original variables.

One further use is principal component analysis on a correlation matrix, obtained with cor=T, which equalizes marginal variances and answers a different scaling-sensitive question. Another is principal component regression: fit a response to selected principal component scores rather than highly correlated original predictors, choosing the number using prediction assessment. The fitted transformation also projects new measurements into the same low-dimensional coordinates, provided the training centering and scaling are retained.

Take logarithms entry by entry before centering. For tree $i$, let $z_i=(\log G_i,\log H_i,\log V_i{)}^T$, and put $\bar{z}=31^{-1}{\sum }_i{z}_i$. The command var(log(b)) computes the sample covariance matrix with the unbiased estimator normalizationThus entry $(j,k)$ is the sum of products of the centered log measurements divided by $30$, not the logarithm of the covariance between the original measurements. Its diagonal entries are the three log-variable sample variances, and the off-diagonal entries are their sample covariances. The positive off-diagonal entries indicate positive association of the log measurements. Taking logarithms turns a multiplicative relation such as $V\approx c{G}^{2}H$ into the approximately linear relation $\log V\approx \log c+2\log G+\log H$, which motivates investigating principal components of these transformed observations.

The first output uses the equal-variance, pooled Student's t-test, as shown by its seven statistical degrees of freedom. Its exact model assumes two independent samples from normal distributions, with means ${\mu }_X,{\mu }_Y$ and a common unknown variance ${\sigma }^2$. The hypotheses are $H_0:{\mu }_X={\mu }_Y$ against $H_1:{\mu }_X<{\mu }_Y$. The sample means are $\bar{x}=4.35$ and $\bar{y}=9.88$, and the unbiased within-sample sample variances areThe pooled sample variance and its resulting standard error areHenceTo justify its reference distribution, under the null $(\bar{X}-\bar{Y})/(\sigma \sqrt{1/4+1/5})$ is standard normal, while $(3s_X^2+4s_Y^2)/{\sigma }^2$ is ${\chi }_7^2$, independently of the two sample means. Dividing the first variable by the square root of the second divided by seven gives the exact pooled two-sample t statistic, with Student t-distribution $t_7$. Its one-sided p-value is the lower tailThe associated one-sided 95% confidence interval for ${\mu }_X-{\mu }_Y$ isThe printed missing lower bound means that this is a one-sided upper confidence limit, not that a finite lower endpoint should be estimated. With such small samples the normality and common-variance assumptions matter; a Welch analysis would use a different variance estimate and generally different degrees of freedom.

The second output is the Wilcoxon rank sum test for two independent continuous samples. Under a common-distribution null hypothesis, labels are exchangeable conditional on the pooled values, so the four ranks assigned to the first sample form a uniformly chosen four-element subset of $\{1,\dots ,9\}$. The first sample receives ranks $2,1,3,5$, giving raw rank sumEquivalently the Mann–Whitney U test statistic is $U_X=W_X-4\cdot 5/2=1$, the number of cross-sample pairs with an $X$ observation larger than a $Y$ observation. This output reports the raw rank sum, rather than the shifted statistic used by some software.

A first sample tending to be smaller has a small rank sum. Under the null, all $\left(\genfrac{}{}{0ex}{}{9}{4}\right)=126$ rank allocations have equal probability. Only $\{1,2,3,4\}$ and $\{1,2,3,5\}$ have rank sum at most 11. By the exact rank-sum tail one above the minimum, the exact lower-tail p-value isNo large-sample normal approximation or continuity correction is involved; the absence of ties makes this simple enumeration valid. For a common-shape location-shift family, the test's alternative is a negative location shift of $X$ relative to $Y$. Without that model, the null is equality of distributions and the rank comparison is not simply a test of equality of means or medians.

Both tests reject at the 5% level in the direction of smaller $X$ observations; only the pooled t-test rejects at 1%. The t-test uses the numerical spacing and targets a mean difference under its normal model. The rank-sum test uses ordering and an exact exchangeability calculation, avoiding the normality and common-normal-variance assumptions. Their different p-values therefore reflect both different statistics and different modelling assumptions, not a numerical contradiction.

The empirical distribution function is ${\hat{F}}_n(x)=n^{-1}{\sum }_i{1}_{X_i\le x}$. For continuous F, the probability integral transform gives independent uniform variables $U_i=F(X_i)$. Strict increase of F is unnecessary. More explicitly, generate the observations by $X_i=F^{-1}(U_i)$ with the generalized quantile inverse. Then $1_{X_i\le x}=1_{U_i\le F(x)}$, so ${\hat{F}}_n(x)=G_n(F(x))$, where $G_n$ is the uniform empirical distribution.

Continuity and the limits of F ensure that its range contains $(0,1)$. The endpoint discrepancies for $G_n$ are zero. Hencewhich depends only on n. Equivalently it is ${\max }_{1\le i\le n}\{i/n-U_{(i)},U_{(i)}-(i-1)/n\}$, a function of uniform order statistics. This proves the distribution-free property of the Kolmogorov-Smirnov statistic even when F has flat portions.

Choose $c_{n,\alpha }$ from this common distribution so that $\mathbb{P}(D_n\le c_{n,\alpha })=1-\alpha$. The Kolmogorov-Smirnov confidence band isThe event that all these inequalities hold is exactly the event used to choose the critical value, giving simultaneous coverage $1-\alpha$. The Glivenko-Cantelli theorem states that for an independent identically distributed sample from any distribution function, continuous or not, ${\sup }_x|{\hat{F}}_n(x)-F(x)|\to 0$ almost surely.

For an integrable r-argument U-statistic kernel, replace it by $h_s(x_1,\dots ,x_r)=(r!{)}^{-1}{\sum }_{\pi }h(x_{\pi (1)},\dots ,x_{\pi (r)})$. Every permutation has the same expectation by exchangeability, so this symmetrization of a U-statistic kernel preserves the target functional. For a symmetric U-statistic kernel h and $n\ge r$, the U-statistic isEach summand has expectation theta, hence the statistic is unbiased.

For the population variance, use $h(x,y)=(x-y{)}^2/2$. Independence gives $\mathbb{E}(X_1-X_2{)}^2=2\mathrm{Var}(X_1)$ when the second moment is finite. The corresponding statistic simplifies using ${\sum }_{i<j}(X_i-X_j{)}^2=n{\sum }_i(X_i-\overline{X}{)}^2$:Thus variance as a U-statistic is precisely the usual unbiased sample variance, for $n\ge 2$.

For the sample, put $\bar{x}=n^{-1}{\sum }_i{x}_i$ and use the sample covariance matrixOrder its eigenvalues ${\hat{\lambda }}_j$ decreasingly and choose unit orthogonal eigenvectors ${\hat{\ell }}_j$. The sample principal components are the score columnsTheir sample means are zero and their sample covariance is diagonal, since ${\hat{\ell }}_j^{T}S{\hat{\ell }}_k={\hat{\lambda }}_j{1}_{\{j=k\}}$. The $j$th sample variance is ${\hat{\lambda }}_j$, and its explained variance of a principal component is ${\hat{\lambda }}_j/\mathrm{tr}(S)$. Using divisor $n$ instead changes the numerical variances by a common factor but leaves the component directions unchanged.

Standardizing means replacing a variable with positive sample variance by $(x_{ij}-{\bar{x}}_j)/s_j$. This gives principal component analysis on a correlation matrix, using $C=D^{-1/2}S{D}^{-1/2}$ with $D=\mathrm{diag}(S)$. It removes arbitrary differences of measurement units and prevents a large-variance coordinate from dominating solely through its scale. It also changes the question being optimized: small-variance and possibly noisy coordinates receive equal marginal weight. Principal component analysis based on the covariance matrix can be more appropriate when the variables have common meaningful units and absolute variability matters. Standardization is therefore a substantive choice, not an automatic improvement. A zero-variance variable must first be removed. In the word-rating example the variables share the same numerical range, but their across-word dispersions can differ; standardization gives each attribute equal initial variance.

For the standardized eight-variable analysis, total variance is $\mathrm{tr}(C)=8$. The first three fractions of explained variance of a principal component are approximately 59.6%, 19.1% and 10.1%, respectively. The first two account for about 78.8%, and the first three for 88.9%. The first direction is dominant but one direction alone loses substantial variation. Retaining two principal components is a plausible descriptive summary: their eigenvalues exceed $1$, whereas the third is below $1$. That cutoff is a heuristic, and the third principal component may still have interpretable content.

The reported coefficient rows are not unit-length vectors. The printed rounded rows have squared lengths $4.7514$, $1.5222$ and $0.8063$, consistent with their eigenvalues to the precision of the coefficient table. They are consistent with the scaled principal component loading convention $r_j=\sqrt{{\lambda }_j}{\ell }_j$. Such rows remain eigenvectors, but the score direction with unit length is ${\ell }_j=r_j/\sqrt{{\lambda }_j}$, or more accurately the row divided by its actual Euclidean norm when using rounded entries. Under this convention $r_{ij}$ is the correlation of standardized attribute $i$ with the variance-one score for component $j$:This explains both their sizes and why they must not be treated as the unit vectors from parts (i) and (ii).

The first principal component has positive coefficients on every attribute, strongest on friendliness, goodness, niceness, bravery and strength. With the favorable endpoints coded in the positive direction, it describes a broad favorable or impressive evaluation, with a potency contribution. The second contrasts movement and speed, with some size and strength, against the first three evaluative attributes; it is chiefly an activity dimension. The third is dominated by size, with smaller positive strength and negative movement/speed coefficients, distinguishing size or potency from activity. These are interpretations of directions of variation, not evidence of causal latent traits. Reversing a principal component's overall sign changes neither its meaning as an axis nor its explained variance; endpoint coding fixes only how its positive side is described.

Write $x_r,x_t$ for two rows of the data matrix and $\delta =x_r-x_t$. Three possible dissimilarities for continuous variables are the following.

The Euclidean distance isIt is simple and has the usual geometric interpretation when coordinates have comparable units. Its disadvantage is sensitivity to the numerical scales: a change of units or a high-variance variable can dominate it. Strongly correlated coordinates can also give repeated weight to essentially the same information. Squared Euclidean distance is useful in some methods, but is not itself a metric because it need not satisfy the triangle inequality.

The standardized Euclidean distance iswhere $s_j^2$ is the sample variance of coordinate $j$ across individuals. It measures differences in standard-deviation units and is invariant under positive coordinate rescalings when the scales are estimated consistently. It is useful for differently measured variables, but requires nonzero sample variances, ignores correlations between coordinates and can give excessive influence to a low-variance noisy variable. With fixed positive scales it is a Euclidean distance after a diagonal change of coordinates, so it is a metric.

The Mahalanobis distance iswhere $S$ is the pooled sample covariance matrix of all individuals. If $S=C{C}^T$ is its Cholesky decomposition, then $d_M=\Vert C^{-1}\delta {\Vert }_2$: it is Euclidean distance in whitened coordinates. It adjusts for both scales and correlations, avoiding duplicate weight for highly correlated variables, and is invariant under nonsingular affine transformations when the sample covariance matrix is transformed accordingly. No multivariate normal distribution assumption is required merely to define the distance. However, $S$ must be a positive-definite matrix; when there are too few observations or dependent coordinates, its inverse is unavailable. Estimation of $S^{-1}$ can also be unstable and sensitive to outliers. These trade-offs determine whether covariance adjustment is preferable to the simpler distances.

Set $Y_i=X_i^2$, $\overline{Y}=n^{-1}{\sum }_i{Y}_i$ and ${\tau }^2={\mathrm{Var}}_F(Y_1)={\mathbb{E}}_F{X}_1^4-{\theta }^2$. The fourth-moment assumption makes ${\tau }^2$ finite, so the variance of a sample mean is $v={\tau }^2/n$. Assume $n\ge 2$, as required for leave-one-out estimation.

The Jackknife variance estimator recomputes the original statistic on each sample with one observation removed. HereBy the definition of the Jackknife variance estimator,For completeness, expanding the centered sum givesConsequently $\mathbb{E}{\hat{v}}_{\mathrm{JACK}}={\tau }^2/n=v$. The jackknife variance estimator is unbiased for this statistic. This calculation uses only the finite fourth moment, not normality of the original observations.

Given the observed data, put $y_i=x_i^2$ and $\overline{y}=n^{-1}{\sum }_i{y}_i$. The empirical distribution assigns mass $1/n$ to each observed $x_i$, including repeated values with their multiplicities. A bootstrap sample consists of $n$ independent draws from this distribution, with replacement. The nonparametric bootstrap variance estimate is the conditional varianceIndependence of the bootstrap draws gives the exact conditional bootstrap variance of a sample mean:Thus its finite-sample expectation is $(n-1)v/n$, in contrast to the unbiased jackknife estimate. This is the jackknife and bootstrap variance of a sample second moment scaling difference.

To approximate it by Monte Carlo simulation, independently generate $B$ bootstrap samples. For replicate $b$, choose $n$ independent uniform indices $I_{b1},\dots ,I_{bn}\in \{1,\dots ,n\}$ and compute $T_b^{\ast }=n^{-1}{\sum }_j{x}_{I_{bj}}^2$. With ${\overline{T}}^{\ast }=B^{-1}{\sum }_b{T}_b^{\ast }$, useFor $B>1$ this is conditionally unbiased for the bootstrap variance and converges to it as $B\to \infty$. Uniform indices can be obtained by $I=\lceil nU\rceil$ for uniforms on $(0,1]$.

Let $G_{\ast }(t)=P_{\ast }(T^{\ast }\le t)$ be the conditional bootstrap distribution of the statistic. A nominal level $1-\alpha$ percentile bootstrap confidence interval isReuse the bootstrap replicates, sort them as $T_{(1)}^{\ast }\le \cdots \le T_{(B)}^{\ast }$, and take the order statistics with ranks $\lceil B\alpha /2\rceil$ and $\lceil B(1-\alpha /2)\rceil$ as empirical quantiles. These are quantiles of the statistic itself, rather than reflected quantiles of its centered error. Increasing $B$ reduces simulation error; it does not turn an approximate confidence interval into an exact finite-sample one.

An analytic alternative follows from the central limit theorem. If ${\tau }^2>0$, then $\sqrt{n}({\hat{\theta }}^n-\theta )\Rightarrow N(0,{\tau }^2)$, and the sample variance $s_Y^2=(n-1{)}^{-1}{\sum }_i(x_i^2-\overline{y}{)}^2$ converges to ${\tau }^2$ by the strong law of large numbers. The Slutsky theorem therefore gives the large-sample normal approximation intervalwhere $z_{1-\alpha /2}$ is the standard normal quantile. Its lower endpoint may be truncated to zero because the parameter is nonnegative. If ${\tau }^2=0$, $X_i^2=\theta$ almost surely and the statistic identifies the parameter exactly; the interval degenerates to that value. The nondegenerate normal interval is asymptotic, not an exact Student interval under arbitrary $F$.

The first matrix contains the leave-one-out samples, and each element of vect is their sample mean. For $n\ge 2$,Thus R4a computes the jackknife variance estimator of the full sample mean:where $s_x^2$ is the unbiased sample variance. R5a computes the estimated bias used in jackknife bias correction. The identity for the average leave-one-out sample mean gives R5a = 0, apart from floating-point rounding.