= Jackknife bias correction
{title2=$\widetilde T_J=nT_n-(n-1)\overline T_J$}
The <jackknife bias correction> uses <jackknife resampling> to estimate and subtract the <bias of an estimator>. Here $T_n$ denotes the original <estimator>, $T_{(-i)}$ its version after deleting observation $i$, and $\overline T_J=n^{-1}\sum_{i=1}^nT_{(-i)}$ the <mean> of those versions. For <sample size> $n>1$, define
$$
\widehat B_n=(n-1)(\overline T_J-T_n),\qquad
\widetilde T_J=T_n-\widehat B_n=nT_n-(n-1)\overline T_J.
$$
The sign matters: a negative estimated <bias> increases the corrected value. The original <estimator> and all deletion versions must target the same <estimand> and use the same estimation rule at their respective <sample sizes>. The definition alone does not make $\widetilde T_J$ an <unbiased estimator>.
For an exact worked example, take the <empirical variance>
$$
T_n=\frac1n\sum_{i=1}^n(X_i-\overline X_n)^2.
$$
On $(1,2,3,4,10)$, the <sample mean> is $4$ and $T_5=50/5=10$. Each deletion <empirical variance> is recomputed around its own deletion <sample mean>, with divisor $4$, giving
$$
(T_{(-1)},\ldots,T_{(-5)})
=(9.6875,11.25,12.1875,12.5,1.25),\qquad
\overline T_J=9.375.
$$
For instance, deleting $10$ leaves $(1,2,3,4)$, whose <sample mean> is $2.5$ and whose <empirical variance> is $1.25$. The correction on the full <statistical sample> is
$$
\widehat B_5=4(9.375-10)=-2.5,\qquad
\widetilde T_J=10-(-2.5)=12.5.
$$
The same data thus give a <jackknife variance estimator> of $2.5$ for the <sample mean>, and a corrected <empirical variance> of $12.5$; these are different <estimands>.
\Image[/jackknife-variance-bias.png]
{title=Two uses of the same deletions: <sampling variance> and <jackknife bias correction>}
{description=The left panel plots contributions to the <jackknife variance estimator> of the <sample mean>, summing to $2.5$. The right panel plots the five deletion <empirical variances> and their <mean> $9.375$, then compares the original <empirical variance> $10$ with the corrected <sample variance> $12.5$.}
The exact equality with the usual <sample variance> is not an accident of these numbers. For any data with $n>1$, write $Q=\sum_i(x_i-\overline x)^2=nT_n$. Removing $x_i$ changes this sum to $Q-n(x_i-\overline x)^2/(n-1)$, so
$$
T_{(-i)}=\frac{Q}{n-1}-\frac{n(x_i-\overline x)^2}{(n-1)^2},\qquad
\overline T_J=\frac{n(n-2)}{(n-1)^2}T_n,\qquad
\widetilde T_J=\frac{n}{n-1}T_n=s_n^2.
$$
For <independent and identically distributed random variables> with finite second <moment> and population <variance> $\sigma^2$, the <expectation> is $\mathbb E T_n=(1-1/n)\sigma^2$. Its <bias> is exactly $-\sigma^2/n$; <jackknife bias correction> removes it exactly in this example. Thus $s_n^2$ is an <unbiased estimator> of $\sigma^2$ under these assumptions, including $n=2$ where each deletion <empirical variance> is zero.
More generally, suppose the <bias> has a regular expansion at both <sample sizes> $n$ and $n-1$, with coefficients $a,b$ independent of $n$:
$$
B_n=\mathbb E T_n-\theta=\frac an+\frac b{n^2}+O(n^{-3}).
$$
Under <independent and identically distributed> sampling, each deletion <estimator> has the same <expectation> as $T_{n-1}$. Hence the corrected <bias> is
$$
\mathbb E\widetilde T_J-\theta=nB_n-(n-1)B_{n-1}=O(n^{-2}),
\qquad n\frac an-(n-1)\frac a{n-1}=0.
$$
The leading $O(n^{-1})$ term cancels; the remaining <bias> need not vanish. The $O$ notation is <big O notation>. This argument presumes the expansion, rather than proving it for every <statistic>; the regularity conditions discussed for the <jackknife variance estimator> and <statistical dependence> still require care. Shalizi's https://stat.cmu.edu/~cshalizi/490/23/2023-11-29-jackknife.pdf[jackknife notes, Section 5.1]{ref} develop the cancellation argument and cite Quenouille's https://academic.oup.com/biomet/article-abstract/43/3-4/353/345190[Notes on Bias in Estimation, Biometrika 43(3–4), 353–360 (1956)]{ref}.
Back to article page
