= Bootstrapping
{disambiguate=statistics}
{wiki}
= Bootstrap
{synonym}
= Bootstrapping
{synonym}
The <bootstrap> approximates a <sampling distribution> by repeatedly <sampling with replacement> from the <empirical measure> of the observed data. In the ordinary nonparametric version, observations are assumed <independent and identically distributed>; a <bootstrap sample> has the same size as the original <statistical sample>. A <statistic>, such as a <sample mean> or <sample median>, is recomputed on each resample. Its variation across resamples estimates how the original <statistic> would vary across repeated sampling from the unknown <probability distribution>.<UW STAT 535: Statistical Machine Learning, Autumn 2026/Lecture 12: The Bootstrap>[UW STAT 535: Statistical Machine Learning, Autumn 2026; The Bootstrap, resampling procedure; High level idea of how bootstrap works, conditional empirical sampling law]{ref}
The key distinction is between the observed <statistical sample> and its still-unknown source <probability distribution>. Condition on the observed values $x_1,\ldots,x_n$ and form the <empirical measure>
$$
\widehat P_n=\frac1n\sum_{i=1}^n\delta_{x_i}.
$$
Here $\delta_{x_i}$ is a <Dirac measure>. Draw <independent> indices $I_1,\ldots,I_n$ from the <uniform distribution on a finite set> $\{1,\ldots,n\}$ and set $X_j^*=x_{I_j}$. Each observed slot has <probability> $1/n$ on every draw. If equal values occupy several slots, their total <probability> is the sum of those slot <probabilities>; distinct observed values need not be equally likely. A star denotes this conditional resampling experiment. There are $n^n$ equally likely ordered index sequences, although <bootstrap count vectors> and <statistics> can combine many such sequences.
\Image[/bootstrap-resampling-observed-slots.png]
{title=<Sampling with replacement> reuses observed slots while keeping the original data fixed}
{description=Original schematic for observations $(1,2,4)$. Three possible ordered <bootstrap samples> have different <sample means>. Their colors identify source slots; none of these samples is new evidence about the unknown <probability distribution>.}
A complicated <sampling distribution> need not stop <statistical inference> when recomputing the <statistic> is easy. The <bootstrap> reuses that calculation on simulated datasets. The <plug-in estimator> principle replaces the unknown <probability measure> by $\widehat P_n$ in the whole repeated-sampling experiment. Whether that substitution approximates the target <sampling distribution> needs an appropriate <bootstrap consistency theorem for the sample mean> or a result for the particular <statistic>. A larger number of resamples reduces <Monte Carlo error>; it cannot repair the <bootstrap failure for a uniform endpoint>, an unrepresentative <statistical sample> or ignored <statistical dependence>.<UW STAT 535: Statistical Machine Learning, Autumn 2026/Lecture 12: The Bootstrap>[UW STAT 535: Statistical Machine Learning, Autumn 2026; High level idea of how bootstrap works; Bootstrap and Statistical Functionals, conditional variance and bootstrap variance consistency; Failure of the bootstrap, uniform-endpoint counterexample]{ref}
Back to article page
