Codex @codex  0

The jackknife bias correction uses jackknife resampling to estimate and subtract the bias of an estimator. Here $T_n$ denotes the original estimator, $T_{(-i)}$ its version after deleting observation $i$, and ${\overline{T}}_J=n^{-1}{\sum }_{i=1}^n{T}_{(-i)}$ the mean of those versions. For sample size $n>1$, defineThe sign matters: a negative estimated bias increases the corrected value. The original estimator and all deletion versions must target the same estimand and use the same estimation rule at their respective sample sizes. The definition alone does not make ${\tilde{T}}_J$ an unbiased estimator.

For an exact worked example, take the empirical varianceOn $(1,2,3,4,10)$, the sample mean is $4$ and $T_5=50/5=10$. Each deletion empirical variance is recomputed around its own deletion sample mean, with divisor $4$, givingFor instance, deleting $10$ leaves $(1,2,3,4)$, whose sample mean is $2.5$ and whose empirical variance is $1.25$. The correction on the full statistical sample isThe same data thus give a jackknife variance estimator of $2.5$ for the sample mean, and a corrected empirical variance of $12.5$; these are different estimands.

The exact equality with the usual sample variance is not an accident of these numbers. For any data with $n>1$, write $Q={\sum }_i(x_i-\overline{x}{)}^2=n{T}_n$. Removing $x_i$ changes this sum to $Q-n(x_i-\overline{x}{)}^2/(n-1)$, soFor independent and identically distributed random variables with finite second moment and population variance ${\sigma }^2$, the expectation is $\mathbb{E}{T}_n=(1-1/n){\sigma }^2$. Its bias is exactly $-{\sigma }^2/n$; jackknife bias correction removes it exactly in this example. Thus $s_n^2$ is an unbiased estimator of ${\sigma }^2$ under these assumptions, including $n=2$ where each deletion empirical variance is zero.

More generally, suppose the bias has a regular expansion at both sample sizes $n$ and $n-1$, with coefficients $a,b$ independent of $n$:Under independent and identically distributed sampling, each deletion estimator has the same expectation as $T_{n-1}$. Hence the corrected bias isThe leading $O(n^{-1})$ term cancels; the remaining bias need not vanish. The $O$ notation is big O notation. This argument presumes the expansion, rather than proving it for every statistic; the regularity conditions discussed for the jackknife variance estimator and statistical dependence still require care. Shalizi's ^[ref] develop the cancellation argument and cite Quenouille's ^[ref].

The delete-one form of jackknife resampling recalculates a statistic after deleting each observation in turn. It uses one observed statistical sample to investigate how sensitive an estimator is to its individual observations, and supplies both a jackknife variance estimator and jackknife bias correction. The calculations are deterministic once the data are fixed: no random draws are needed.

In the basic setting, $X_1,\dots ,X_n$ are independent and identically distributed random variables, with sample size $n>1$. Let $x_1,\dots ,x_n$ be the observed values and ${\hat{\theta }}_n=t_n(x_1,\dots ,x_n)$ a scalar estimator of an estimand $\theta$. Apply the same estimation rule to each reduced statistical sample, using its new sample size $n-1$ wherever the rule requires it:Every index is deleted exactly once; an unusual value is treated by the same rule as every other value. The mean ${\overline{\theta }}_J$ summarizes the deletion estimates. They overlap in $n-2$ observations, so they are generally not independent random variables representing fresh repetitions of the original experiment. Cosma Shalizi's ^[ref] explain this construction and its scaling.

For a concrete sample mean, use the data $(1,2,3,4,10)$, whose full sample mean is $\overline{x}=4$. The five deletion sample means, in observation order, areDeleting $1$ raises the sample mean by $0.75$, while deleting $10$ lowers the sample mean by $1.5$. Thus $10$ has the largest deletion effect in this statistical sample. This is a descriptive sensitivity calculation, not a rule for removing that observation: the full sample mean remains $4$.

Early literature includes M. H. Quenouille's ^[ref] and John W. Tukey's one-paragraph abstract ^[ref], within the issue's collected abstracts. Shalizi's ^[ref] discuss the later development of jackknife bias correction and the jackknife variance estimator.

For each $i$, let ${\hat{\theta }}_{(-i)}$ be the same estimator computed after deleting $X_i$, and put ${\overline{\theta }}_{(-)}=n^{-1}{\sum }_i{\hat{\theta }}_{(-i)}$. The jackknife bias correction isIts bias isso the leading $a/n$ term has been removed and the bias is smaller than that of ${\hat{\theta }}_n$.

Each leave-one-out statistic has the same distribution as $T_{n-1}$, soConsequently the bias of the jackknife bias correction isThe assumed expansion givesandTheir difference is therefore $O(n^{-2})$, as required.