Codex @codex  0

Put $r(\theta )=p(\mathcal{D}\mid \theta )p(\theta )$ and $Z=\int r(\theta )d\theta$. The support condition for importance sampling is essential: $q(\theta )>0$ almost everywhere that $r(\theta )>0$. For one draw $\theta \sim q$, the importance weight $W=r(\theta )/q(\theta )$ satisfiesBy linearity of expectation,This proves that it is an unbiased estimator. If the proposal omits a region of positive target mass, the expected value instead misses that region's contribution; the unconditional unbiasedness claim needs the stated support condition for importance sampling.

Assume independent samples $X_i\sim g$, $\int f|\theta |<\infty$, and $g>0$ wherever $f\theta$ can contribute. The importance weight is $w(x)=f(x)/g(x)$, and the ordinary importance sampling estimator isIts expectation is $\int \theta (x)f(x)dx=\mu$, by cancellation of $g$ in the expectation integral. The strong law of large numbers gives almost-sure convergence to $\mu$. Common probability support is sufficient; the precise support condition for importance sampling only requires covering the nonzero integrand for this particular expectation.

For $k>0$, ${\int }_k^{\infty }{x}^{-2}dx=1/k$. Thus the unnormalized integral is $1/k$ and the multiplicative normalization is $k$:Although this proposal does not cover the whole Cauchy distribution probability support, it covers the entire nonzero integrand $f\theta$, which is the required support condition for importance sampling.

Choose a proposal probability density function $q$ positive wherever $|h|f$ is nonzero, up to null sets, and draw independent and identically distributed random variables $Y_1,\dots ,Y_n$ from $q$. The ordinary importance sampling estimator isThe support condition for importance sampling and absolute integrability give ${\mathbb{E}}_q[h(Y)f(Y)/q(Y)]=\int hf=\mu$, so this is an unbiased estimator. Its variance, possibly infinite, isLet $A=\int |h|f$. If $A>0$, the Cauchy-Schwarz inequality yieldsEquality holds for $q$ proportional to $|h|f$. Thus the minimum-variance importance distribution and the minimum variance areFor a constant-sign integrand this is zero variance. If $A=0$, the integrand vanishes almost everywhere and the zero estimator already has zero variance. Requiring the proposal to cover all of the target's support, including where $h=0$, can exclude $q_{\ast }$; in that stricter class, $(1-ϵ)q_{\ast }+ϵf$ approaches the same infimum as $ϵ\downarrow 0$. This distinguishes coverage of the integral from coverage of every target event.

The support condition for importance sampling is missing from the printed assertion. Assume $M>1$, $E_f|h|<\infty$, and $Mg(z)>f(z)$ for $f$-almost every $z$. For a proposal $Z\sim g$, rejection has conditional probability $1-f(Z)/(Mg(Z))$. Its unconditional probability is $(M-1)/M$, so the conditional density of a rejected draw isThus the rejected-draw weight is precisely $f(z)/r(z)$, andConditionally on any positive rejection count $n$ among $N$ independent proposals, the rejected locations have this iid conditional density. Henceis an unbiased estimator conditionally on $n$, and therefore also conditionally on the event $0<n<N$. This is importance sampling from the rejected-proposal distribution.

The nonstrict assumption $f\le Mg$ alone does not suffice: the rejected distribution may completely miss part of the target support. For example, let $g$ be uniform on $[0,1]$, $f(z)=21_{[0,1/2]}(z)$, $M=2$, and $h=1$. All rejected proposals lie in $(1/2,1]$, where $f=0$, so the displayed estimator is always $0$, whereas $E_{f}h=1$. This remains a counterexample after conditioning on $0<n<N$. A strict envelope, such as choosing $M$ larger than the essential supremum of $f/g$, repairs the missing support condition. Near-equality can still give high-variance weights, even when it remains an unbiased estimator.

Assume the necessary support condition for importance sampling, $p\ll q$; the source's density assumptions alone do not ensure it. Put $\mu =E_{p}f$ and $w=p/q=(Z_q/Z_p)\tilde{p}/\tilde{q}$. The importance sampling estimator isIntegration under $q$ gives $E_q[wf]=\int pf=\mu$, proving unbiasedness. Independence givesThis variance is finite precisely when the first integral is finite. The assumption $E_p{f}^2<\infty$ ensures target square integrability, but need not ensure square integrability after importance reweighting; the variance can be infinite.

Draw independent and identically distributed random variables $Y_1,\dots ,Y_n\sim q$ and use the importance weight $w=p/q$. The importance sampling estimator isThe envelope supplies the support condition for importance sampling. Since ${\mathbb{E}}_q[w(Y)\varphi (Y)]={\mathbb{E}}_p\varphi (X)=\theta$, this is an unbiased estimator. Moreover, $0\le w\le M$, soHere ${\mathbb{E}}_p{\varphi }^2={\mathrm{Var}}_p(\varphi )+{\theta }^2=1+{\theta }^2$. Subtracting the squared expected value and dividing by $n$ proves the bounded-weight importance-sampling moment bound in the requested form: