Bootstrap sample

A bootstrap sample of size $n$ is drawn with replacement from the $n$ observed data points. Some observations occur repeatedly and about a proportion $e^{-1}$ are omitted.

For independent mean-zero errors in a fixed-design regression, form $y_i^{\ast }=x_i^{\top }\hat{\beta }+{\hat{e}}_i{W}_i$, where the independent multipliers have mean zero and variance one, and refit. Rademacher multipliers taking the values $-1$ and $1$ with equal probability are a simple choice. Conditional on the data, the bootstrap errors have mean zero and preserve observation-specific residual variances, supporting heteroscedasticity rather than imposing a common residual distribution. Appropriate leverage corrections and regularity assumptions are required for accurate studentized inference.

For independent identically distributed pairs $(X_i,Y_i)$, resample whole pairs using the same index for both coordinates. This preserves the empirical within-pair dependence needed to estimate a correlation coefficient or another joint statistic. Independently resampling the two margins destroys that dependence and estimates a different sampling law.

Conditionally on the observed values, bootstrap draws are iid from their empirical distribution. Each draw has variance $n^{-1}{\sum }_i(a_i-\overline{a}{)}^2$, so averaging $n$ independent draws gives the displayed variance. The variance across $B$ independent replicate means, using denominator $B-1$, is unbiased for this conditional quantity. Its expectation over the original iid data is $(n-1)\mathrm{Var}(a_1)/n^2$, which is slightly smaller than the actual sampling variance $\mathrm{Var}(a_1)/n$.

An unordered nonparametric bootstrap sample from $n$ distinct observations is specified by its $n$ nonnegative multiplicities. Stars and bars gives $\left(\genfrac{}{}{0ex}{}{2n-1}{n}\right)$ such vectors. They are not equiprobable: a vector has probability $n!/(n^n{\prod }_i{N}_i!)$, because this counts its ordered sequences of sampled indices.