Past exam of the mathematics course of the University of Cambridge / 2008 / iii / Paper 47 / 4

For jackknife resampling, calculate the leave-one-out estimates ${\hat{\theta }}_{(-i)}$ by deleting observation $i$, and their average ${\overline{\theta }}_J=n^{-1}{\sum }_i{\hat{\theta }}_{(-i)}$. The jackknife variance estimator isThe factor $n-1$ compensates for the small differences between the overlapping samples. For sufficiently smooth statistics it estimates their sampling variance. The related estimated bias is $(n-1)({\overline{\theta }}_J-\hat{\theta })$, leading to the jackknife bias correction $n\hat{\theta }-(n-1){\overline{\theta }}_J$.

For the nonparametric bootstrap, replace the unknown distribution by its empirical distribution functionIndependently sample $n$ observations with replacement from the original observations, compute the estimator on that bootstrap sample, and repeat this for $B$ independently generated bootstrap samples. If the resulting values are ${\hat{\theta }}_1^{\ast },\dots ,{\hat{\theta }}_B^{\ast }$, estimate the conditional bootstrap variance byIts interpretation as an approximation to the original sampling variance relies on the bootstrap approximating the sampling law of the statistic; increasing $B$ only reduces simulation error and cannot repair a failure of that approximation.

Since the original observations are distinct, a bootstrap sample has no repeated value precisely when it contains all $n$ original values once. Of the $n^n$ equally likely ordered resamples, $n!$ are permutations without repetition. ThereforeThe probability is zero for $n=1$ and rapidly approaches one as $n$ grows.

The first matrix contains the leave-one-out samples, and each element of vect is their sample mean. For $n\ge 2$,Thus R4a computes the jackknife variance estimator of the full sample mean:where $s_x^2$ is the unbiased sample variance. R5a computes the estimated bias used in jackknife bias correction. The identity for the average leave-one-out sample mean gives R5a = 0, apart from floating-point rounding.

The second printed loop contains an indexing error: its row index is i, although its loop variable is b. Literally, after the first block, i equals $n$. When $n\le 199$, every iteration overwrites row $n$ of the new matrix; the other rows remain missing. Hence vect contains one finite mean and 198 missing entries. R's default sorting removes missing entries, leaving a vector of length one, so selecting its fifth and 195th entries gives R7b = c(NA, NA). If $n>199$, the matrix assignment is out of bounds; if the second block runs independently with i undefined, it fails even earlier. This is a source error, not a property of the bootstrap.

Replacing just the row index by b gives the intended algorithm: draw 199 independent bootstrap samples and sort their sample means as $s_1\le \cdots \le s_{199}$. The two indices are $200(0.05/2)=5$ and $200(1-0.05/2)=195$, soThese are the endpoints of an approximate 95% percentile bootstrap confidence interval for the population mean. Their actual numerical values depend on the supplied data and random draws, which are not specified. They are not the reflected endpoints of a basic bootstrap confidence interval, and the choice of ranks does not guarantee exact finite-sample coverage.

Negative covariance can be exploited through antithetic variates, but it must belong to an actual joint sampling scheme. Two separately independent samples do not inherit negative covariance merely because one can imagine a coupling of their marginal distributions. Interpret the data as $h=\min (n,m)$ independent matched pairs $(X_i,Y_i)$ with covariance ${\sigma }_{XY}<0$, together with independent unmatched observations if the sample sizes differ. Other known pairing schemes can be handled by substituting their actual covariance of the two means.

Let $A=\overline{X}$, $B=\overline{Y}$, and putFor any fixed $w$, the combination $wA+(1-w)B$ is an unbiased estimator of the common mean. Its variance is $w^2{V}_X+(1-w{)}^2{V}_Y+2w(1-w)C$. Differentiation, or completing the square, gives the minimum-variance combination of unbiased sample means:With $C<0$, both weights are positive. This is efficiency among fixed linear unbiased combinations, not a universal claim about every possible estimator for arbitrary distributions $F,G$.

For an immediately implementable exactly unbiased estimator without known moments, use fixed $w=n/(n+m)$, the pooled sample mean. Its variance iswhich shows directly the gain from negative covariance. For any fixed weight, estimate $V_X,V_Y$ using $s_X^2/n,s_Y^2/m$, and $C$ using $h{s}_{XY}/(nm)$, with $s_{XY}$ the usual unbiased sample covariance of the matched pairs. Substitute them into the variance quadratic. At least two matched pairs are needed to estimate their covariance nonparametrically.

If the unknown optimal weights are desired while retaining exact unbiasedness, a cross-fitted combination of unbiased sample means gives an algorithm using the supplied data. Split the independent pairs and unmatched observations into two folds, keeping each pair together and retaining observations of both types in each fold. Estimate the moments from fold 2 to choose the weight for the fold-1 sample means, adjusting the variance terms for fold 1's sample sizes; do the converse for fold 2. Clip estimated weights to $(0\le w\le 1)$, to keep them bounded when training moment estimates are unstable. Average the two held-out combinations with deterministic weights, for example one half each for balanced folds.

Conditional on its training fold, each held-out pair of sample means has expectation $\theta$, and its weights sum to one. Each held-out combination, and hence their average, is therefore exactly an unbiased estimator. Under consistent moment estimation with increasing balanced folds, the weights approach $w_{\ast }$ and recover its first-order variance advantage. In contrast, estimating a weight from the same data to which it is applied need not yield an unbiased estimator.

Estimate the variance of this fitted-weight procedure by a bootstrap that resamples matched pairs as blocks and unmatched observations separately, repeats the entire split/weight/combination procedure, and takes the sample variance of the resulting estimates. This preserves the negative covariance and includes variability of the fitted weights. Resampling the $x$ and $y$ values independently would erase the feature responsible for the improvement. The bootstrap estimate has its usual large-sample qualification; the fixed-weight plug-in variance formula above is the simpler alternative when weights are predetermined.