Monte Carlo estimator

A Monte Carlo estimator is a random approximation obtained from simulated samples, commonly a sample average or a weighted sample average.

For $n$ observations, taking the mean of their scalar sum returns that sum, not their sample mean. If $a-b\overline{H}$ is unbiased for a target, replacing the mean by a sum gives expected value $a-bn\mathbb{E}H$ and multiplies its variance by $n^2$. A mean-zero control variate changes variance but cannot correct this missing normalization factor.

At equal proposal count, compare $\int {\varphi }^2{f}^2/g-{\theta }^2$ for unnormalized importance sampling with $M(\int {\varphi }^{2}f-{\theta }^2)$ for the accepted-sample mean. There is no universal ordering for nonzero target mean. For zero mean, the bounded-weight inequality gives importance sampling no larger asymptotic variance. Empty rejected samples and per-proposal computational cost are separate considerations.

An additive constant leaves the target variance and accepted-sample mean variance unchanged, but changes the variance of unnormalized importance sampling because its random weights do not sum to one. With g uniform on (0,1), f(x)=2x and M=2, phi(x)=x gives importance variance 16/45 versus rejection proposal-budget variance 1/9. Replacing phi by x-1 gives importance variance 1/45 with the same rejection variance, reversing preference.

A control variate is a random variable with known expected value whose centered value is subtracted from a Monte Carlo estimator to reduce its variance. For a mean-zero vector $G$, put $S=\mathrm{Cov}(G)$ and $c=\mathrm{Cov}(G,H)$. When $S$ is invertible, the optimal estimator averages $H-c^T{S}^{-1}G$ and has variance $(\mathrm{Var}(H)-c^T{S}^{-1}c)/N$ for $N$ independent draws. Improvement is strict exactly when $c\ne 0$.

If $Y$ is an unbiased estimator and a vector $C$ has known mean, then $Y-{\beta }^T(C-\mathbb{E}C)$ remains unbiased for a deterministic coefficient vector. Its variance is $\mathrm{Var}Y-2{\beta }^{T}c+{\beta }^T\Sigma \beta$, where $c=\mathrm{Cov}(C,Y)$ and $\Sigma =\mathrm{Cov}(C)$. For invertible $\Sigma$, completing the square gives the displayed optimum and minimum variance $\mathrm{Var}Y-c^T{\Sigma }^{-1}c$. With singular $\Sigma$, use a generalized inverse on its range. Estimating coefficients from the same simulation can introduce finite-sample bias; fixed pilot coefficients or independent training preserve the elementary unbiasedness assertion.

For $Y$ uniform on $(0,2)$, put $g(y)=1/(\pi (1+y^2))$ and $H(y)=y^2-4/3$. Then $\mathbb{E}H=0$, and $1/2-2\overline{g-\beta H}$ estimates the standard Cauchy distribution probability above two. Its variance is $4\mathrm{Var}(g-\beta H)/n$, minimized by the displayed coefficient. Here $\mathrm{Var}(H)=64/45$ and $\mathrm{Cov}(g,H)=1/\pi -7\arctan (2)/(6\pi )$, so the optimal coefficient is negative. A coefficient estimated using a separate pilot can be fixed for a fresh unbiased production sample.

If a smooth posterior density has vanishing boundary terms, its log-posterior gradient satisfies $\mathbb{E}[g]=0$ andThis is a posterior integration-by-parts identity; it concerns differentiation in the random parameter, distinct from the usual mean-zero score identity for sampling distributions. It makes $g$ a vector control variate. Finite moments and known coefficients preserve unbiasedness of the Monte Carlo estimator.