Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 47 / 4

The nonparametric bootstrap replaces the unknown sampling law by the empirical distributionGenerate $B$ independent bootstrap samples, each containing $n$ independent draws with replacement from that law. Recompute the original estimator in each sample, obtaining ${\hat{\theta }}_1^{\ast },\dots ,{\hat{\theta }}_B^{\ast }$. With ${\overline{\theta }}^{\ast }=B^{-1}{\sum }_b{\hat{\theta }}_b^{\ast }$, the bootstrap standard error estimate isThis estimates the conditional bootstrap standard deviation, which approximates the estimator's sampling standard deviation when the bootstrap is consistent. Dividing this expression by $\sqrt{B}$ would instead estimate the Monte Carlo error in the bootstrap mean, a different quantity.

For distinct original observations, each resample is an ordered index sequence with one of $n^n$ equally likely values. It contains no repeats precisely when it is a permutation of all $n$ original indices, giving $n!$ possibilities. HenceThe distinctness assumption is needed to equate repeated values with repeated indices.

Calculate the observed sample correlation coefficient $\hat{r}$. In each paired bootstrap, draw 100 indices independently with replacement from $1,\dots ,100$ and select the corresponding whole pairs. Compute the bootstrap correlation $r_b^{\ast }$ from each resample; do not resample the two margins independently, since that would destroy their empirical dependence.

Let $q_p$ be the empirical $p$-quantile of those bootstrap correlations. The percentile bootstrap confidence interval isThis is the requested algorithmic interval; the actual endpoints require the observed pairs. Its coverage is approximate, based on the sampling law of the correlation being well approximated by the bootstrap law. Both sample variances must be positive. Resamples having a constant margin have undefined correlation and cannot be repaired by arbitrarily assigning correlation zero. One may regenerate these exceptional resamples, explicitly conditioning on well-defined correlations; under the usual nondegenerate large-sample conditions their probability is negligible. Frequent degeneracy instead indicates that this ordinary correlation-bootstrap construction is unsuitable.

For a bootstrap-t confidence interval, estimate the original standard error $\hat{s}$ of $\hat{r}$, for example by the standard deviation of the outer paired bootstrap correlations. For each outer resample $b$, estimate its own standard error $s_b^{\ast }$ by drawing many inner paired resamples from that outer sample and taking the standard deviation of their correlations. The resulting studentized statistics areLet $t_p^{\ast }$ be their empirical quantiles. Approximating the distribution of $(\hat{r}-r)/\hat{s}$ by that of $T^{\ast }$ and solving the two inequalities for $r$ givesThe reversal of quantiles is essential. The pivot requires positive standard errors and nondegenerate correlations; its claimed 95% coverage is an approximation under bootstrap regularity, not an exact finite-sample guarantee. One may intersect the final interval with the parameter space $[-1,1]$.

The same percentile and studentized procedures can be applied to the sample mean of $X$, and then only the $X$ observations need be resampled. They are useful when a normal approximation is doubtful. However the mean already has the simple standard-error estimate $S_X/10$ here. For normal observations, the classical exact interval isand the same form is an asymptotic approximation for independent finite-variance observations under appropriate large-sample conditions. Thus bootstrap intervals are valid alternatives, but are often unnecessary for a mean with a reliable classical pivot; the correlation has no corresponding distribution-free elementary pivot. An infinite-variance distribution would require other theory rather than this ordinary standard-error argument.

A control variate is a random variable $C$ correlated with the estimator and with known mean $c_0$. For a fixed real coefficient $a$, define $Y_a=Y-a(C-c_0)$. Then $\mathbb{E}{Y}_a=\theta$, whileAssuming finite second moments and positive control variance, complete the square to findWhen $\mathrm{Var}(Y)>0$, the minimum is $\mathrm{Var}(Y)(1-{\rho }_{Y,C}^2)$. A constant control cannot reduce variance. A strongly correlated control, including a negatively correlated one with a negative coefficient, can substantially reduce it.

For multivariate control variates, write $Y_{\beta }=Y-{\beta }^T(C-\mathbb{E}C)$, $\Sigma =\mathrm{Cov}(C)$ and $c=\mathrm{Cov}(C,Y)$. Completing the multivariate square givesIf $\Sigma$ is singular, use its Moore-Penrose inverse. Indeed any null direction has a constant centered control combination and therefore zero covariance with $Y$, so $c$ lies in the range of $\Sigma$. The elementary unbiasedness statement assumes a deterministic coefficient; estimating it from the same simulation may introduce finite-sample bias, whereas an independently trained coefficient keeps the conditional unbiasedness argument valid.