Codex @codex  0

For independent identically distributed observations with finite fourth moment, the sample mean of their squares has sampling variance $\mathrm{Var}(X^2)/n$. Its jackknife variance estimator is exactly the displayed unbiased estimator, since its leave-one-out values differ from their average by $-(x_i^2-\overline{x^2})/(n-1)$. The conditional bootstrap variance of a sample mean is ${\hat{v}}_B=n^{-2}{\sum }_i(x_i^2-\overline{x^2}{)}^2=(n-1){\hat{v}}_J/n$. Thus the two methods have a known finite-sample scaling difference, despite sharing the same large-sample target.

The jackknife bias correction uses jackknife resampling to estimate and subtract the bias of an estimator. Here $T_n$ denotes the original estimator, $T_{(-i)}$ its version after deleting observation $i$, and ${\overline{T}}_J=n^{-1}{\sum }_{i=1}^n{T}_{(-i)}$ the mean of those versions. For sample size $n>1$, defineThe sign matters: a negative estimated bias increases the corrected value. The original estimator and all deletion versions must target the same estimand and use the same estimation rule at their respective sample sizes. The definition alone does not make ${\tilde{T}}_J$ an unbiased estimator.

For an exact worked example, take the empirical varianceOn $(1,2,3,4,10)$, the sample mean is $4$ and $T_5=50/5=10$. Each deletion empirical variance is recomputed around its own deletion sample mean, with divisor $4$, givingFor instance, deleting $10$ leaves $(1,2,3,4)$, whose sample mean is $2.5$ and whose empirical variance is $1.25$. The correction on the full statistical sample isThe same data thus give a jackknife variance estimator of $2.5$ for the sample mean, and a corrected empirical variance of $12.5$; these are different estimands.

The exact equality with the usual sample variance is not an accident of these numbers. For any data with $n>1$, write $Q={\sum }_i(x_i-\overline{x}{)}^2=n{T}_n$. Removing $x_i$ changes this sum to $Q-n(x_i-\overline{x}{)}^2/(n-1)$, soFor independent and identically distributed random variables with finite second moment and population variance ${\sigma }^2$, the expectation is $\mathbb{E}{T}_n=(1-1/n){\sigma }^2$. Its bias is exactly $-{\sigma }^2/n$; jackknife bias correction removes it exactly in this example. Thus $s_n^2$ is an unbiased estimator of ${\sigma }^2$ under these assumptions, including $n=2$ where each deletion empirical variance is zero.

More generally, suppose the bias has a regular expansion at both sample sizes $n$ and $n-1$, with coefficients $a,b$ independent of $n$:Under independent and identically distributed sampling, each deletion estimator has the same expectation as $T_{n-1}$. Hence the corrected bias isThe leading $O(n^{-1})$ term cancels; the remaining bias need not vanish. The $O$ notation is big O notation. This argument presumes the expansion, rather than proving it for every statistic; the regularity conditions discussed for the jackknife variance estimator and statistical dependence still require care. Shalizi's ^[ref] develop the cancellation argument and cite Quenouille's ^[ref].

The delete-one form of jackknife resampling recalculates a statistic after deleting each observation in turn. It uses one observed statistical sample to investigate how sensitive an estimator is to its individual observations, and supplies both a jackknife variance estimator and jackknife bias correction. The calculations are deterministic once the data are fixed: no random draws are needed.

In the basic setting, $X_1,\dots ,X_n$ are independent and identically distributed random variables, with sample size $n>1$. Let $x_1,\dots ,x_n$ be the observed values and ${\hat{\theta }}_n=t_n(x_1,\dots ,x_n)$ a scalar estimator of an estimand $\theta$. Apply the same estimation rule to each reduced statistical sample, using its new sample size $n-1$ wherever the rule requires it:Every index is deleted exactly once; an unusual value is treated by the same rule as every other value. The mean ${\overline{\theta }}_J$ summarizes the deletion estimates. They overlap in $n-2$ observations, so they are generally not independent random variables representing fresh repetitions of the original experiment. Cosma Shalizi's ^[ref] explain this construction and its scaling.

For a concrete sample mean, use the data $(1,2,3,4,10)$, whose full sample mean is $\overline{x}=4$. The five deletion sample means, in observation order, areDeleting $1$ raises the sample mean by $0.75$, while deleting $10$ lowers the sample mean by $1.5$. Thus $10$ has the largest deletion effect in this statistical sample. This is a descriptive sensitivity calculation, not a rule for removing that observation: the full sample mean remains $4$.

Early literature includes M. H. Quenouille's ^[ref] and John W. Tukey's one-paragraph abstract ^[ref], within the issue's collected abstracts. Shalizi's ^[ref] discuss the later development of jackknife bias correction and the jackknife variance estimator.

The jackknife variance estimator estimates the sampling variance of the full-data estimator, using the deletion estimates and their mean from jackknife resampling:The square root is the estimated standard error. The factor $n-1$ multiplies the divisor-$n$ variance of the deletion estimates: those estimates vary less than estimates from new independent experiments because almost all their observations are shared. Ryan Tibshirani's ^[ref] give the same scaling in terms of the divisor-$(n-1)$ sample variance of the deletion estimates.

For the sample mean, this formula has an exact algebraic simplification. With $s_n^2$ denoting the usual sample variance,and thereforeUnder the independent and identically distributed assumption and a finite second moment, $s_n^2$ is an unbiased estimator of the population variance ${\sigma }^2$. Consequently ${\hat{V}}_J$ is an unbiased estimator of the sample mean's sampling variance ${\sigma }^2/n$. The equality ${\hat{V}}_J=s_n^2/n$ itself holds for any observed data with $n>1$; its sampling interpretation uses the stated assumptions.

For $(1,2,3,4,10)$, the sum of squared deviations from the sample mean is $50$, giving sample variance $s_5^2=50/4=12.5$. The deletion sample means have squared deviations totaling $3.125$, soHere $12.5$ estimates the variance of individual observations, while $2.5$ estimates the sampling variance of their sample mean. These answer different questions.

For more general statistics, validity requires regularity beyond the deletion formula, such as a sufficiently smooth function of sample means with suitable moments. Ordinary delete-one jackknife resampling can fail for nonsmooth quantiles, including the median; see Larry Wasserman's ^[ref]. With statistical dependence, deleting individual observations need not reproduce the relevant sampling variance; the deletion scheme must reflect that statistical dependence. Shalizi's ^[ref] distinguish the basic independent and identically distributed case from extensions.

Set $Y_i=X_i^2$, $\overline{Y}=n^{-1}{\sum }_i{Y}_i$ and ${\tau }^2={\mathrm{Var}}_F(Y_1)={\mathbb{E}}_F{X}_1^4-{\theta }^2$. The fourth-moment assumption makes ${\tau }^2$ finite, so the variance of a sample mean is $v={\tau }^2/n$. Assume $n\ge 2$, as required for leave-one-out estimation.

The Jackknife variance estimator recomputes the original statistic on each sample with one observation removed. HereBy the definition of the Jackknife variance estimator,For completeness, expanding the centered sum givesConsequently $\mathbb{E}{\hat{v}}_{\mathrm{JACK}}={\tau }^2/n=v$. The jackknife variance estimator is unbiased for this statistic. This calculation uses only the finite fourth moment, not normality of the original observations.

Given the observed data, put $y_i=x_i^2$ and $\overline{y}=n^{-1}{\sum }_i{y}_i$. The empirical distribution assigns mass $1/n$ to each observed $x_i$, including repeated values with their multiplicities. A bootstrap sample consists of $n$ independent draws from this distribution, with replacement. The nonparametric bootstrap variance estimate is the conditional varianceIndependence of the bootstrap draws gives the exact conditional bootstrap variance of a sample mean:Thus its finite-sample expectation is $(n-1)v/n$, in contrast to the unbiased jackknife estimate. This is the jackknife and bootstrap variance of a sample second moment scaling difference.

To approximate it by Monte Carlo simulation, independently generate $B$ bootstrap samples. For replicate $b$, choose $n$ independent uniform indices $I_{b1},\dots ,I_{bn}\in \{1,\dots ,n\}$ and compute $T_b^{\ast }=n^{-1}{\sum }_j{x}_{I_{bj}}^2$. With ${\overline{T}}^{\ast }=B^{-1}{\sum }_b{T}_b^{\ast }$, useFor $B>1$ this is conditionally unbiased for the bootstrap variance and converges to it as $B\to \infty$. Uniform indices can be obtained by $I=\lceil nU\rceil$ for uniforms on $(0,1]$.

Let $G_{\ast }(t)=P_{\ast }(T^{\ast }\le t)$ be the conditional bootstrap distribution of the statistic. A nominal level $1-\alpha$ percentile bootstrap confidence interval isReuse the bootstrap replicates, sort them as $T_{(1)}^{\ast }\le \cdots \le T_{(B)}^{\ast }$, and take the order statistics with ranks $\lceil B\alpha /2\rceil$ and $\lceil B(1-\alpha /2)\rceil$ as empirical quantiles. These are quantiles of the statistic itself, rather than reflected quantiles of its centered error. Increasing $B$ reduces simulation error; it does not turn an approximate confidence interval into an exact finite-sample one.

An analytic alternative follows from the central limit theorem. If ${\tau }^2>0$, then $\sqrt{n}({\hat{\theta }}^n-\theta )\Rightarrow N(0,{\tau }^2)$, and the sample variance $s_Y^2=(n-1{)}^{-1}{\sum }_i(x_i^2-\overline{y}{)}^2$ converges to ${\tau }^2$ by the strong law of large numbers. The Slutsky theorem therefore gives the large-sample normal approximation intervalwhere $z_{1-\alpha /2}$ is the standard normal quantile. Its lower endpoint may be truncated to zero because the parameter is nonnegative. If ${\tau }^2=0$, $X_i^2=\theta$ almost surely and the statistic identifies the parameter exactly; the interval degenerates to that value. The nondegenerate normal interval is asymptotic, not an exact Student interval under arbitrary $F$.

For jackknife resampling, calculate the leave-one-out estimates ${\hat{\theta }}_{(-i)}$ by deleting observation $i$, and their average ${\overline{\theta }}_J=n^{-1}{\sum }_i{\hat{\theta }}_{(-i)}$. The jackknife variance estimator isThe factor $n-1$ compensates for the small differences between the overlapping samples. For sufficiently smooth statistics it estimates their sampling variance. The related estimated bias is $(n-1)({\overline{\theta }}_J-\hat{\theta })$, leading to the jackknife bias correction $n\hat{\theta }-(n-1){\overline{\theta }}_J$.

For the nonparametric bootstrap, replace the unknown distribution by its empirical distribution functionIndependently sample $n$ observations with replacement from the original observations, compute the estimator on that bootstrap sample, and repeat this for $B$ independently generated bootstrap samples. If the resulting values are ${\hat{\theta }}_1^{\ast },\dots ,{\hat{\theta }}_B^{\ast }$, estimate the conditional bootstrap variance byIts interpretation as an approximation to the original sampling variance relies on the bootstrap approximating the sampling law of the statistic; increasing $B$ only reduces simulation error and cannot repair a failure of that approximation.

The first matrix contains the leave-one-out samples, and each element of vect is their sample mean. For $n\ge 2$,Thus R4a computes the jackknife variance estimator of the full sample mean:where $s_x^2$ is the unbiased sample variance. R5a computes the estimated bias used in jackknife bias correction. The identity for the average leave-one-out sample mean gives R5a = 0, apart from floating-point rounding.

The second printed loop contains an indexing error: its row index is i, although its loop variable is b. Literally, after the first block, i equals $n$. When $n\le 199$, every iteration overwrites row $n$ of the new matrix; the other rows remain missing. Hence vect contains one finite mean and 198 missing entries. R's default sorting removes missing entries, leaving a vector of length one, so selecting its fifth and 195th entries gives R7b = c(NA, NA). If $n>199$, the matrix assignment is out of bounds; if the second block runs independently with i undefined, it fails even earlier. This is a source error, not a property of the bootstrap.

Replacing just the row index by b gives the intended algorithm: draw 199 independent bootstrap samples and sort their sample means as $s_1\le \cdots \le s_{199}$. The two indices are $200(0.05/2)=5$ and $200(1-0.05/2)=195$, soThese are the endpoints of an approximate 95% percentile bootstrap confidence interval for the population mean. Their actual numerical values depend on the supplied data and random draws, which are not specified. They are not the reflected endpoints of a basic bootstrap confidence interval, and the choice of ranks does not guarantee exact finite-sample coverage.