Control variates

A control variate is a random variable with known expected value whose centered value is subtracted from a Monte Carlo estimator to reduce its variance. For a mean-zero vector $G$, put $S=\mathrm{Cov}(G)$ and $c=\mathrm{Cov}(G,H)$. When $S$ is invertible, the optimal estimator averages $H-c^T{S}^{-1}G$ and has variance $(\mathrm{Var}(H)-c^T{S}^{-1}c)/N$ for $N$ independent draws. Improvement is strict exactly when $c\ne 0$.

If $Y$ is an unbiased estimator and a vector $C$ has known mean, then $Y-{\beta }^T(C-\mathbb{E}C)$ remains unbiased for a deterministic coefficient vector. Its variance is $\mathrm{Var}Y-2{\beta }^{T}c+{\beta }^T\Sigma \beta$, where $c=\mathrm{Cov}(C,Y)$ and $\Sigma =\mathrm{Cov}(C)$. For invertible $\Sigma$, completing the square gives the displayed optimum and minimum variance $\mathrm{Var}Y-c^T{\Sigma }^{-1}c$. With singular $\Sigma$, use a generalized inverse on its range. Estimating coefficients from the same simulation can introduce finite-sample bias; fixed pilot coefficients or independent training preserve the elementary unbiasedness assertion.

For $Y$ uniform on $(0,2)$, put $g(y)=1/(\pi (1+y^2))$ and $H(y)=y^2-4/3$. Then $\mathbb{E}H=0$, and $1/2-2\overline{g-\beta H}$ estimates the standard Cauchy distribution probability above two. Its variance is $4\mathrm{Var}(g-\beta H)/n$, minimized by the displayed coefficient. Here $\mathrm{Var}(H)=64/45$ and $\mathrm{Cov}(g,H)=1/\pi -7\arctan (2)/(6\pi )$, so the optimal coefficient is negative. A coefficient estimated using a separate pilot can be fixed for a fresh unbiased production sample.

If a smooth posterior density has vanishing boundary terms, its log-posterior gradient satisfies $\mathbb{E}[g]=0$ andThis is a posterior integration-by-parts identity; it concerns differentiation in the random parameter, distinct from the usual mean-zero score identity for sampling distributions. It makes $g$ a vector control variate. Finite moments and known coefficients preserve unbiasedness of the Monte Carlo estimator.