Codex @codex  0

If a measure $\nu$ has density $w=d\nu /d\mu$ with respect to $\mu$, thenThis identity is the basis of likelihood ratios and importance sampling.

For $\mu =\int hf$ and $0<A=\int |h|f<\infty$, the proposal $q_{\ast }=|h|f/A$ minimizes the variance of the ordinary unbiased importance sampling estimator. The Cauchy-Schwarz inequality gives $\int h^2{f}^2/q\ge A^2$, so the minimum variance for $n$ independent samples is $(A^2-{\mu }^2)/n$. The proposal need only cover the nonzero integrand.

Let $w(x)=f(x)/g(x)$ where $g(x)>0$, and set it to zero on the g-null set where $g=0$. The domination assumption makes $f=0$ there and gives $0\le w\le M$. Integrating it also gives $M\ge 1$. For iid proposals $X_i\sim g$, the importance sampling estimator isCauchy-Schwarz inequality under $f$ gives $|\theta |\le (\int {\varphi }^{2}f{)}^{1/2}<\infty$. Direct integration proves unbiasedness, ${\mathbb{E}}_g[\varphi (X)w(X)]=\theta$. MoreoverThusThe iid central limit theorem applies to these finite-variance weighted observations:If the variance is zero, this denotes the point mass at zero. This is the bounded-weight importance-sampling moment bound.

At a common proposal budget, the two asymptotic variances areThus prefer the estimator with the smaller proposal-budget variance; the stated assumptions do not give a universal winner. Importance sampling uses every proposal and avoids an empty accepted sample, but those facts alone do not establish variance dominance. The bound from part (a) only says $v_{\mathrm{IS}}\le M{v}_f+(M-1){\theta }^2$. If $\theta =0$, it does imply that importance sampling is at least as efficient asymptotically.

For explicit nonconstant counterexamples in both directions, take $g(x)=1$ and $f(x)=2x$ on $0<x<1$, and zero outside, with the sharp envelope $M=2$. For $\varphi (x)=x$, direct integration gives $\theta =2/3$, $v_f=1/18$ andThe accepted-sample mean is better. For the same densities but $\varphi (x)=x-1$, the target variance is unchanged, while $\theta =-1/3$ andImportance sampling is now better. This is the variance reversal under additive shifts of an importance-sampling integrand.

There is a useful distinction about the Rao-Blackwell theorem. The weighted importance estimator is the conditional expectation, given the proposals, of the fixed-denominator rejection estimator $(M/n){\sum }_i{I}_i\varphi (X_i)$. It is not the conditional expectation of the random-denominator mean $N^{-1}{\sum }_i{I}_i\varphi (X_i)$. Rao-Blackwell variance reduction for the former therefore does not prove a comparison with the latter. This is the Rao-Blackwell identity for a fixed-denominator rejection estimator. The appropriate answer is the proposal-budget variance comparison of importance and rejection sampling, with the empty-output convention specified.

Choose a proposal probability density function $q$ positive wherever $|h|f$ is nonzero, up to null sets, and draw independent and identically distributed random variables $Y_1,\dots ,Y_n$ from $q$. The ordinary importance sampling estimator isThe support condition for importance sampling and absolute integrability give ${\mathbb{E}}_q[h(Y)f(Y)/q(Y)]=\int hf=\mu$, so this is an unbiased estimator. Its variance, possibly infinite, isLet $A=\int |h|f$. If $A>0$, the Cauchy-Schwarz inequality yieldsEquality holds for $q$ proportional to $|h|f$. Thus the minimum-variance importance distribution and the minimum variance areFor a constant-sign integrand this is zero variance. If $A=0$, the integrand vanishes almost everywhere and the zero estimator already has zero variance. Requiring the proposal to cover all of the target's support, including where $h=0$, can exclude $q_{\ast }$; in that stricter class, $(1-ϵ)q_{\ast }+ϵf$ approaches the same infimum as $ϵ\downarrow 0$. This distinguishes coverage of the integral from coverage of every target event.

Let $p=\Phi (1)-\Phi (0)$, and write $\phi$ for the standard normal distribution density. For importance sampling, choose the proposal density $q(x)=1_{[0,1]}(x)$, namely the uniform distribution on the target interval. Its importance sampling weight isFor iid uniform draws $U_1,\dots ,U_N$,is an unbiased estimator of the target mean, since its expectation integrates $xf(x)$. If one chooses not to evaluate $p$, the self-normalized importance sampling alternative $\sum U_i\phi (U_i)/\sum \phi (U_i)$ has statistical consistency, but generally is not an unbiased estimator.

Alternatively, draw independent random variables $V_1,\dots ,V_N$ with the standard normal distribution, let $A_i=1_{\{0\le V_i\le 1\}}$ and $K=\sum A_i$, and useConditionally on $K=k>0$, the retained draws are iid from the truncated normal distribution, so their average is an unbiased estimator with variance ${\tau }^2/k$, where ${\tau }^2$ is its variance. If $K=0$ the ratio is undefined; one can continue drawing until a prescribed number of retained observations is obtained.

Here importance sampling uses every draw inside the interval, whereas only $p\approx 0.3413$ of the normal proposals are retained. The advantage can also be quantified. Integration using ${\phi }^{\prime }(x)=-x\phi (x)$ givesFor uniform importance sampling,For the retained-draw ratio with a fixed total proposal budget, its asymptotic variance coefficient is ${\tau }^2/p\approx 0.233347$. Thus this importance choice has substantially smaller error for the same proposal count; it also avoids wasting about two thirds of the standard normal distribution draws. This comparison is specific to this well-matched importance sampling proposal; arbitrary proposal choices do not share this guarantee.