Monte Carlo integration (${\hat{I}}_S=S^{-1}{\sum }_{s=1}^{S}h(X_s)$)

To calculate $I=\int h(x)p(x)dx$, generate independent $X_s$ with density $p$ and average $h(X_s)$. Integrability gives convergence by the strong law of large numbers; finite variance gives an unbiased estimate with variance $\mathrm{Var}(h(X))/S$. Applying this to indicators estimates probabilities. Correlated Markov chain Monte Carlo samples can also approximate the integral, but their error is determined by the chain's dependence rather than the independent-sample variance formula.

Antithetic variates deliberately couple unbiased estimators of the same expectation with negative covariance. Their average can have smaller variance than an average using independent draws. For example, evaluating a monotone integrand at $U$ and $1-U$, where $U$ has a uniform distribution, gives opposite fluctuations while preserving both marginal distributions.

For standard Cauchy distribution probability density function $f$ and $k>0$, this estimates its upper-tail probability without sampling a heavy tail. The function $f(ku)$ decreases on $[0,1]$, while $f(k(1-u))$ increases, so their covariance is nonpositive. The paired estimator is unbiased and has no larger variance than an estimator using the same number $2r$ of independent probability density function evaluations.

For two unbiased estimators $A,B$ of the same parameter with variances $V_X,V_Y$ and covariance $C$, the fixed-weight estimator $wA+(1-w)B$ is unbiased and has variance $w^2{V}_X+(1-w{)}^2{V}_Y+2w(1-w)C$. Completing the square gives $w_{\ast }=(V_Y-C)/(V_X+V_Y-2C)$ and minimum variance $(V_X{V}_Y-C^2)/(V_X+V_Y-2C)$ when the denominator is positive. These are optimal among fixed linear unbiased combinations. Substituting weights estimated from the same observations need not preserve unbiasedness.

Split independent observational blocks into two folds. Use one fold to estimate the combination weight for the two sample means in the other fold, and conversely. Each held-out combination is an unbiased estimator conditionally on its training fold because its two held-out sample means have the same expectation. A deterministic average of the two combinations remains unbiased. Under consistency of the moment estimates and balanced fold sizes, the weights approach those of the minimum-variance combination of unbiased sample means. Paired observations must stay together when splitting or bootstrapping so their covariance is preserved.