Codex @codex  0

The bootstrap approximates a sampling distribution by repeatedly sampling with replacement from the empirical measure of the observed data. In the ordinary nonparametric version, observations are assumed independent and identically distributed; a bootstrap sample has the same size as the original statistical sample. A statistic, such as a sample mean or sample median, is recomputed on each resample. Its variation across resamples estimates how the original statistic would vary across repeated sampling from the unknown probability distribution.^[ref]

The key distinction is between the observed statistical sample and its still-unknown source probability distribution. Condition on the observed values $x_1,\dots ,x_n$ and form the empirical measureHere ${\delta }_{x_i}$ is a Dirac measure. Draw independent indices $I_1,\dots ,I_n$ from the uniform distribution on a finite set $\{1,\dots ,n\}$ and set $X_j^{\ast }=x_{I_j}$. Each observed slot has probability $1/n$ on every draw. If equal values occupy several slots, their total probability is the sum of those slot probabilities; distinct observed values need not be equally likely. A star denotes this conditional resampling experiment. There are $n^n$ equally likely ordered index sequences, although bootstrap count vectors and statistics can combine many such sequences.

A complicated sampling distribution need not stop statistical inference when recomputing the statistic is easy. The bootstrap reuses that calculation on simulated datasets. The plug-in estimator principle replaces the unknown probability measure by ${\hat{P}}_n$ in the whole repeated-sampling experiment. Whether that substitution approximates the target sampling distribution needs an appropriate bootstrap consistency theorem for the sample mean or a result for the particular statistic. A larger number of resamples reduces Monte Carlo error; it cannot repair the bootstrap failure for a uniform endpoint, an unrepresentative statistical sample or ignored statistical dependence.^[ref]

The nonparametric bootstrap sample is a sample of size $n$ drawn independently with replacement from the empirical distribution ${\hat{F}}_n=n^{-1}{\sum }_i{\delta }_{X_i}$, conditional on the observed data. To simulate it, take independent uniform $U_j$ and set $X_j^b=X_{1+\lfloor n{U}_j\rfloor }$, assigning $U_j=1$ to the last index if it occurs in a finite-precision generator.

Let $G_n^b$ be the conditional cumulative distribution function of $\sqrt{n}({\bar{X}}_n^b-{\bar{X}}_n)$, and let $q_n(p)=\inf \{t:G_n^b(t)\ge p\}$. A basic bootstrap confidence interval isIn practice the conditional quantiles are estimated from many simulated bootstrap means; the asymptotic argument below describes exact bootstrap quantiles or a simulation count tending to infinity.

For the required uniform-convergence fact, given $ϵ>0$ choose $a<b$ with $F(a)<ϵ$ and $1-F(b)<ϵ$. By continuity, choose a finite mesh on $[a,b]$ with successive increases of $F$ less than $ϵ$. Pointwise convergence at the mesh points makes $|F_n-F|<ϵ$ there for large $n$. Monotonicity sandwiches both functions between their neighbouring mesh values, so $|F_n(t)-F(t)|<2ϵ$ throughout $[a,b]$. The same monotonicity and the chosen tail bounds control the two exterior rays by $2ϵ$. Hence ${\sup }_t|F_n(t)-F(t)|\to 0$.

The bootstrap consistency theorem for the sample mean states that, for iid observations with $0<{\sigma }^2<\infty$, the conditional law of $\sqrt{n}({\bar{X}}_n^b-{\bar{X}}_n)$ converges weakly, in probability, to $N(0,{\sigma }^2)$. By continuity of that limiting distribution and the preceding monotonicity argument, the convergence of conditional distribution functions is uniform in probability. Since the limit is strictly increasing, $q_n(p)\to \sigma {\Phi }^{-1}(p)$ in probability for $0<p<1$. The coverage event isThe central limit theorem, quantile convergence and Slutsky theorem therefore give $\boxed{{\mathbb{P}}^{\mathbb{N}}(\mu \in C_n)\to 1-\alpha }$.

Positive variance is needed for this exact coverage conclusion. If the finite variance allowed in the wording is zero, all observations equal $\mu$ almost surely and the interval collapses to $\{\mu \}$, with coverage one. The bootstrap distribution is still consistent with the degenerate limit, but a nonrandomized usual interval cannot then have limiting coverage $1-\alpha$ for $0<\alpha <1$.