Monte Carlo method

A Monte Carlo method approximates a deterministic quantity using averages of simulated random variables and controls the approximation with probabilistic limit theorems.

Unbiased random termination of a sampled contribution: retain it with probability $q>0$ and multiply surviving contributions by $1/q$. In path tracing this controls path length without the systematic truncation of a fixed bounce limit, provided the continuation probabilities and weights are handled consistently.

Stochastic methods for quantum many-particle expectation values or ground-state projection. Variational and diffusion approaches can include correlations beyond a single Slater determinant; fermion signs or nodal constraints and finite-system extrapolation require explicit control.

A pseudorandom number generator is a deterministic state-update algorithm producing numbers intended to imitate an independent sequence from a prescribed probability distribution. For Monte Carlo integration, desirable features include good one-dimensional and multidimensional uniformity, a period far longer than the calculation, negligible detectable dependence, reproducible seeds, sufficient numerical precision and fast generation. Uniform marginal distributions alone do not provide independent draws.

A value initializing a pseudorandom generator. Repeating a seed need not reproduce a parallel stochastic calculation unless the generator, stream assignment and implementation are also controlled.

A pseudorandom number generator whose state sequence combines earlier values at two fixed lags, for example $S_n=(S_{n-j}+S_{n-k})\mathrm{mod}m$ with $0<j<k$. A seed supplies the first $k$ values. A circular buffer of $k$ entries suffices for generation because each update uses only the retained lag window.

A pseudorandom number is an output of a deterministic algorithm chosen to imitate samples from a specified probability law. For example, a linear congruential generator returns fractions $S_i/M$ on a finite grid in $[0,1)$. Their intended uniform distribution is an approximation, and their serial dependence must be assessed separately from the generator's period.

Choose integers $M>1$, $a$, $c$ and a seed $S_0$. Iterate $S_{n+1}=(a{S}_n+c)\mathrm{mod}M$ and return $U_n=S_n/M$. This gives a deterministic finite-state pseudorandom number generator with output in $[0,1)$. A nonzero increment gives a mixed generator; $c=0$ gives a multiplicative generator. Its period is at most $M$, but even a full period does not guarantee good multidimensional uniformity or statistical independence.

A linear congruential generator has period $M$ for every seed if and only if $\gcd (c,M)=1$, every prime dividing $M$ divides $a-1$, and $4\mid M$ implies $4\mid a-1$. The coprimality of $c$ prevents the orbit from being trapped in a residue class; the multiplier conditions permit full-period lifting through each prime power. Combining the prime-power periods by the Chinese remainder theorem gives period $M$. For $M=12^k=2^{2k}{3}^k$, the multiplier criterion reduces to $a\equiv 1(\mathrm{mod}12)$, together with $\gcd (c,6)=1$. A positive increment alone is insufficient.

To calculate $I=\int h(x)p(x)dx$, generate independent $X_s$ with density $p$ and average $h(X_s)$. Integrability gives convergence by the strong law of large numbers; finite variance gives an unbiased estimate with variance $\mathrm{Var}(h(X))/S$. Applying this to indicators estimates probabilities. Correlated Markov chain Monte Carlo samples can also approximate the integral, but their error is determined by the chain's dependence rather than the independent-sample variance formula.

Antithetic variates deliberately couple unbiased estimators of the same expectation with negative covariance. Their average can have smaller variance than an average using independent draws. For example, evaluating a monotone integrand at $U$ and $1-U$, where $U$ has a uniform distribution, gives opposite fluctuations while preserving both marginal distributions.

For standard Cauchy distribution probability density function $f$ and $k>0$, this estimates its upper-tail probability without sampling a heavy tail. The function $f(ku)$ decreases on $[0,1]$, while $f(k(1-u))$ increases, so their covariance is nonpositive. The paired estimator is unbiased and has no larger variance than an estimator using the same number $2r$ of independent probability density function evaluations.

For two unbiased estimators $A,B$ of the same parameter with variances $V_X,V_Y$ and covariance $C$, the fixed-weight estimator $wA+(1-w)B$ is unbiased and has variance $w^2{V}_X+(1-w{)}^2{V}_Y+2w(1-w)C$. Completing the square gives $w_{\ast }=(V_Y-C)/(V_X+V_Y-2C)$ and minimum variance $(V_X{V}_Y-C^2)/(V_X+V_Y-2C)$ when the denominator is positive. These are optimal among fixed linear unbiased combinations. Substituting weights estimated from the same observations need not preserve unbiasedness.

Split independent observational blocks into two folds. Use one fold to estimate the combination weight for the two sample means in the other fold, and conversely. Each held-out combination is an unbiased estimator conditionally on its training fold because its two held-out sample means have the same expectation. A deterministic average of the two combinations remains unbiased. Under consistency of the moment estimates and balanced fold sizes, the weights approach those of the minimum-variance combination of unbiased sample means. Paired observations must stay together when splitting or bootstrapping so their covariance is preserved.

Replacing a simulated quantity $H$ by $E[H\mid \mathcal{G}]$ preserves its expectation and reduces variance by the law of total variance. This is Rao-Blackwellization. For a random sum, conditioning on the count and all but one summand replaces an event indicator by a cumulative distribution function evaluation. The resulting estimator needs that conditional expectation to be computable.

Independent uniform distribution draws $U_1,U_2$ on $(0,1)$ produce independent standard normal distribution draws $X_1=\sqrt{-2\log U_1}\cos (2\pi U_2)$ and $X_2=\sqrt{-2\log U_1}\sin (2\pi U_2)$. The proof transforms an independent uniform angle and radial density $r{e}^{-r^2/2}$ through the polar-coordinate Jacobian.

If $V$ is uniform on $(0,1)$, the displayed random variable is not standard normal. For $V\le 1/2$ it is nonnegative; for $V>1/2$, its radius is at most $\sqrt{2\log 2}$, so it is always bounded below by $-\sqrt{2\log 2}$. A standard normal distribution has an unbounded lower tail.

The Box-Muller transform instead uses a common radius $\sqrt{-2\log U}$ independent of the angle $2\pi V$ in both Cartesian coordinates. Their polar density is $r{e}^{-r^2/2}/(2\pi )$, and dividing by the coordinate Jacobian $r$ gives the product of two standard normal densities. Changing a logarithm's uniform input changes the distribution, even if the resulting expression superficially resembles that formula.

Nested Monte Carlo approximates an outer expected value whose integrand contains another expected value. A nonlinear transformation, such as a logarithm, of an inner sample average usually introduces finite-sample bias. Increasing both sample sizes requires suitable integrability controls to establish consistency; an unbiased estimator of the inner quantity does not make its logarithm unbiased.

A Monte Carlo estimator is a random approximation obtained from simulated samples, commonly a sample average or a weighted sample average.

For $n$ observations, taking the mean of their scalar sum returns that sum, not their sample mean. If $a-b\overline{H}$ is unbiased for a target, replacing the mean by a sum gives expected value $a-bn\mathbb{E}H$ and multiplies its variance by $n^2$. A mean-zero control variate changes variance but cannot correct this missing normalization factor.

At equal proposal count, compare $\int {\varphi }^2{f}^2/g-{\theta }^2$ for unnormalized importance sampling with $M(\int {\varphi }^{2}f-{\theta }^2)$ for the accepted-sample mean. There is no universal ordering for nonzero target mean. For zero mean, the bounded-weight inequality gives importance sampling no larger asymptotic variance. Empty rejected samples and per-proposal computational cost are separate considerations.

An additive constant leaves the target variance and accepted-sample mean variance unchanged, but changes the variance of unnormalized importance sampling because its random weights do not sum to one. With g uniform on (0,1), f(x)=2x and M=2, phi(x)=x gives importance variance 16/45 versus rejection proposal-budget variance 1/9. Replacing phi by x-1 gives importance variance 1/45 with the same rejection variance, reversing preference.

A control variate is a random variable with known expected value whose centered value is subtracted from a Monte Carlo estimator to reduce its variance. For a mean-zero vector $G$, put $S=\mathrm{Cov}(G)$ and $c=\mathrm{Cov}(G,H)$. When $S$ is invertible, the optimal estimator averages $H-c^T{S}^{-1}G$ and has variance $(\mathrm{Var}(H)-c^T{S}^{-1}c)/N$ for $N$ independent draws. Improvement is strict exactly when $c\ne 0$.

If $Y$ is an unbiased estimator and a vector $C$ has known mean, then $Y-{\beta }^T(C-\mathbb{E}C)$ remains unbiased for a deterministic coefficient vector. Its variance is $\mathrm{Var}Y-2{\beta }^{T}c+{\beta }^T\Sigma \beta$, where $c=\mathrm{Cov}(C,Y)$ and $\Sigma =\mathrm{Cov}(C)$. For invertible $\Sigma$, completing the square gives the displayed optimum and minimum variance $\mathrm{Var}Y-c^T{\Sigma }^{-1}c$. With singular $\Sigma$, use a generalized inverse on its range. Estimating coefficients from the same simulation can introduce finite-sample bias; fixed pilot coefficients or independent training preserve the elementary unbiasedness assertion.

For $Y$ uniform on $(0,2)$, put $g(y)=1/(\pi (1+y^2))$ and $H(y)=y^2-4/3$. Then $\mathbb{E}H=0$, and $1/2-2\overline{g-\beta H}$ estimates the standard Cauchy distribution probability above two. Its variance is $4\mathrm{Var}(g-\beta H)/n$, minimized by the displayed coefficient. Here $\mathrm{Var}(H)=64/45$ and $\mathrm{Cov}(g,H)=1/\pi -7\arctan (2)/(6\pi )$, so the optimal coefficient is negative. A coefficient estimated using a separate pilot can be fixed for a fresh unbiased production sample.

If a smooth posterior density has vanishing boundary terms, its log-posterior gradient satisfies $\mathbb{E}[g]=0$ andThis is a posterior integration-by-parts identity; it concerns differentiation in the random parameter, distinct from the usual mean-zero score identity for sampling distributions. It makes $g$ a vector control variate. Finite moments and known coefficients preserve unbiasedness of the Monte Carlo estimator.

Monte Carlo error is the random difference between a simulation estimate and the exact quantity it approximates. For an average of independent finite-variance draws, its standard deviation is proportional to the inverse square root of the number of draws.

The Monte Carlo standard error estimates the standard deviation of a simulation-based estimate across repeated simulation runs. For an average of $N$ independent random variables with finite variance ${\sigma }_h^2$, it is ${\sigma }_h/\sqrt{N}$. Under a suitable central limit theorem for Markov chain Monte Carlo, correlated draws instead give the approximation ${\sigma }_h/\sqrt{N_{\mathrm{eff}}}$, where $N_{\mathrm{eff}}$ is the effective sample size of a Markov chain for the particular estimated function $h$. This measures numerical uncertainty in an estimate, not uncertainty in the unknown parameter itself.

Suppose $f$ is a target probability density and $h$ is a proposal density with $f(x)\le Mh(x)$. Draw $X\sim h$ and an independent $U\sim \mathrm{Uniform}[0,1]$, and accept $X$ whenThe accepted value has density $f$, the acceptance probability is $1/M$, and the expected number of proposals is $M$.

Stopping after a fixed number of rejection sampling proposals preserves the target law of accepted labels, conditional on the number accepted. Stopping instead after a fixed amount of elapsed computing time can change the finite-sample law if processing costs depend on the proposed state. The deadline may censor precisely the expensive states.

For an exact counterexample, let both target and proposal be uniform on two states, so every proposal is accepted. Let processing cost be one time unit for the first state and two for the second. Stop between one and two units and retain only finished proposals. Conditional on obtaining an output, the first state is the only possible value, despite its target probability being one half. This proves that a runtime cutoff need not be an unbiased stopping rule for the realized finite output. It does not imply asymptotic bias of the indefinitely growing completed sequence; it identifies a finite-horizon censoring effect.

Consider independent rejection sampling trials with proposal density $q$, accepted-value density $f$, acceptance probability $a\in (0,1)$ and rejection probability $r=1-a$. Splitting the proposal law by its accept/reject outcome gives $q=af+r{g}_R$, where $g_R$ is the rejected-proposal distribution.

If an algorithm permits $n$ trials, returns the first accepted value when one occurs, and otherwise returns the final rejected proposal, its output density is the displayed mixture. The event of $n$ failures has probability $r^n$ and gives the conditional law $g_R$; every successful stopping trial gives law $f$. Hence the forced return introduces bias unless $g_R=f$, and its contribution decays geometrically with the cap. Conditional on success before any specified trial, the output still has law $f$, since accepted value and successful-trial index are independent.

For a Poisson distribution target of mean $\lambda >0$ and a geometric distribution proposal with probability mass function $q_k=(1-t)t^k$ on the nonnegative integers, the tight rejection sampling envelope isSuccessive target-to-proposal ratios have ratio $\lambda /[t(k+1)]$, proving this maximizing index. The function $\log M(t)$ is the maximum of the convex functions $-\lambda -\log (1-t)+k\log (\lambda /t)-\log (k!)$. Within a branch its derivative is $(1-t{)}^{-1}-m/t$.

Writing $K=\lceil \lambda \rceil$, the optimum is $t_{\ast }=\min \{\lambda /K,K/(K+1)\}$, and the success parameter is ${\rho }_{\ast }=1-t_{\ast }$. For $K-1<\lambda \le K^2/(K+1)$, $t_{\ast }=\lambda /K$ is a branch boundary; the left and right derivatives are nonpositive and nonnegative respectively. For $K^2/(K+1)<\lambda \le K$, the active branch has a zero derivative at $t_{\ast }=K/(K+1)$, including a tie at the upper endpoint. Convexity proves global optimality. The minimum expected proposal count is $M(t_{\ast })$; no claim about machine-specific instruction costs is implied.

Draw proposals with probability density function or probability mass function $q$, independently draw $U$ from the uniform distribution on $(0,1)$, and repeat until $U\le h(Y)$ for a nonnegative threshold $h$. The actual acceptance probability given $Y=y$ is $a(y)=\min (1,h(y))$. If $A={\mathbb{E}}_q[a(Y)]>0$, the returned probability distribution has density or mass $q(y)a(y)/A$. Indeed the joint law of a proposal and acceptance has density or mass $q(y)a(y)$, and conditioning on acceptance divides by $A$. Summing over failed trials gives the same law and expected trial count $1/A$. Thresholds exceeding one cannot be used as unbounded reweighting factors.

A lower envelope $h$ can save evaluations of an expensive unnormalized target $\tilde{p}$ in rejection sampling. Draw $Y$ with probability density function $\tilde{q}/Z_q$ and an independent uniform distribution value $U$. Accept immediately if $U\le h(Y)/(M\tilde{q}(Y))$. Otherwise draw a fresh independent uniform distribution value $V$ and accept ifFor a fixed proposal, writing $B=M\tilde{q}(Y)$, $H=h(Y)$ and $P=\tilde{p}(Y)$ gives total acceptance probability $H/B+(1-H/B)(P-H)/(B-H)=P/B$. If $H=B$, the first stage accepts with probability one and the residual test is skipped. Thus the accepted probability density function is $\tilde{p}/Z_p$, and the overall acceptance probability is $Z_p/(M{Z}_q)$. The probability of avoiding a target evaluation on one proposal is $\int h/(M{Z}_q)$.

If $f\ge 0$, $C=\int f\in (0,\infty )$ and $f(x)\le Mg(x)$ for a proposal probability density function $g$, accept a proposal $Y$ when an independent uniform distribution value is at most $f(Y)/(Mg(Y))$. Integrating the joint acceptance probability density function gives $p_{\mathrm{acc}}=C/M$ and conditional accepted probability density function $f/C$. For $n$ independent proposals the accepted count has binomial distribution with parameters $n,C/M$.

For target density proportional to $x^{\alpha -1}{e}^{-\beta x}$ on $x>a>0$, use the Pareto distribution proposal $g(x)=b{a}^b{x}^{-b-1}$ with $b>0$. Its density ratio is proportional to $x^{\alpha +b}{e}^{-\beta x}$, maximized at $(\alpha +b)/\beta$ if that point exceeds $a$. The tight envelope yields the highest acceptance probability for this fixed proposal. An inverse draw is $a{U}^{-1/b}$ with $U$ uniformly distributed on $(0,1)$.

For $a,b>1$, the maximum of $r$ on $[0,1]$ occurs at $(a-1)/(a+b-2)$. If one parameter is one, the maximum is attained at the corresponding endpoint; if both are one, $r$ is constant. Let $M=\sup r$. Accept an independent uniform distribution proposal $X$ when another uniform draw is at most $r(X)/M$. The accepted probability density function is the beta distribution, and acceptance probability is $B(a,b)/M$.

With proper prior $\pi (\theta )$ independent of the neutral Kingman's coalescent times, propose $(\theta ,T)$ from their joint prior and accept with probability $e^{-\theta L/2}(\theta L/2{)}^k/k!$. The accepted density is proportional to the prior times the observed-count likelihood, hence is the posterior given $S=k$. The acceptance rate is the prior predictive probability $P(S=k)$. For $k>0$, divide the acceptance function by its maximum $M_k=e^{-k}{k}^k/k!$, achieved at $\theta L/2=k$, to improve the rate to $P(S=k)/M_k$ without altering the accepted distribution. For $k=0$, $M_0=1$.

For a proper prior distribution $\pi (\theta )$ independent of coalescent times $T$, let $q(T)={\prod }_{j=2}^n{\lambda }_j{e}^{-{\lambda }_j{T}_j}$ with ${\lambda }_j=\left(\genfrac{}{}{0ex}{}{j}{2}\right)$ and $L=\sum j{T}_j$. Observing $S=k$ under an infinite sites mutation model gives posterior density $\pi (\theta )q(T)e^{-\theta L/2}(\theta L/2{)}^k/(k!m_k)$, where $m_k$ is the prior predictive probability of $S=k$. Multiplication by the conditional Poisson mass and normalization is an application of Bayes' theorem.

// Target: statistical-inference.bigb

Accepting a fresh uniform deck permutation with a probability depending only on whether its endpoint ranks match preserves uniform individual rank marginals by symmetry. It changes their joint law. For four cards of each of thirteen ranks and rejection probability $p$ on a mismatch, the accepted matching probability is $1/(17-16p)$. The two accepted ranks are independent precisely at $p=1/4$; equality of marginals alone does not imply independence.

Write $a=\nu /2$. Against an exponential distribution proposal of mean $\nu$, the chi-squared distribution density has envelope constant $M=a^a{e}^{1-a}/\Gamma (a)$. The density ratio divided by $M$ is $\exp ((a-1)(\log (y/\nu )-y/\nu +1))$. Thus generate $Y=-\nu \log U$ and accept when $\log V\le (a-1)(\log (Y/\nu )-Y/\nu +1)$ for independent uniform distributions. The inequality $\log t\le t-1$ validates the acceptance probability. The accepted probability density function is the target because rejection sampling weights the proposal by its target-to-envelope ratio.

For a nonnegative integrable function $h$ with integral $Z$, its ratio-of-uniforms region has area $Z/2$. A uniform point in the region transforms under $v=ux$ with Jacobian determinant $u$; integrating $2u/Z$ over $0<u\le \sqrt{h(x)}$ gives density $h(x)/Z$. A rectangle containing the region gives a practical rejection sampling algorithm using only uniform random variables, with acceptance $Z/(2\mathrm{area}(B))$.

For the unnormalized Laplace distribution probability density function $h(x)=e^{-|x-1|/2}$, the ratio-of-uniforms method uses $0<u\le \sqrt{h(v/u)}$. The maximum of $h$ is one. On $x\le 0$, the maximum of $x^{2}h(x)$ occurs at $x=-4$ and equals $16e^{-5/2}$. On $x\ge 0$, it occurs at $x=4$ and equals $16e^{-3/2}$. Thus the displayed rectangle contains the region. Its area is $4(e^{-3/4}+e^{-5/4})$, while the target region has area $\frac{1}{2}\int h=2$. Uniform proposals in the rectangle, accepted when $u^2\le h(v/u)$, therefore return the correct Laplace distribution with acceptance probability $[2(e^{-3/4}+e^{-5/4}){]}^{-1}$.

For the unnormalized normal density $h(x)=e^{-x^2/(2{\sigma }^2)}$, the smallest axis-aligned envelope is $0<u<1$ and $|v|\le \sigma \sqrt{2/e}$. The boundary reaches that width at $u=e^{-1/2}$. Region area is $\sigma \sqrt{\pi /2}$, so the acceptance rate is $\sqrt{\pi e}/4$, independently of $\sigma$. Multiplying $h$ by a positive constant rescales both coordinates and their areas consistently.

A finite bounding rectangle exists when $\sqrt{h(x)}$ and $x\sqrt{h(x)}$ are bounded. This is not automatic for every probability density function: tails proportional to $(1+|x|{)}^{-3/2}$ give an unbounded second coordinate. In principle any finite-area measurable region has an open envelope of finite area, decomposable up to null boundaries into disjoint dyadic rectangles. Select a rectangle with probability proportional to its area, then sample its two coordinates uniformly and reject outside the target region. The accepted point is uniform on the region, with acceptance equal to the ratio of their areas. Constructing an effective envelope remains a density-specific computational step.

For the half-normal distribution, the density ratio to $\mathrm{Exp}(\lambda )$ is maximized at $y=\lambda$. Completing the square gives the envelope constant above and acceptance probability $e^{-(y-\lambda {)}^2/2}$. Its logarithmic derivative is $\lambda -1/\lambda$, so the optimal rate is $\lambda =1$. Give an accepted nonnegative value an independent fair sign to obtain the standard normal distribution. The expected number of proposals is $M$, while the probability of rejecting one proposal is $1-1/M$; these are different quantities.

Reject pairs of independent uniform coordinates from the square until their squared radius $S$ lies in $(0,1)$. Multiply each coordinate by $\sqrt{-2\log S/S}$ to obtain two independent variables with the standard normal distribution. Squaring either gives a chi-squared variable with one degree of freedom.

For n independent proposals and an envelope constant $M$, each rejection decision succeeds with probability $1/M$. The accepted count is Bin(n,1/M). Conditional on any acceptance pattern, the accepted values have independent target density f; summing over patterns preserves that product law conditional on the count. The empty-output probability is $(1-1/M{)}^n$.

For rejection sampling with acceptance probability $1/M$, $M>1$, the accepted count in $n$ trials has binomial distribution, mean $n/M$ and variance $n(M-1)/M^2$. To obtain at least $N$ acceptances with approximate probability $1-\delta$, use the standard normal distribution quantile $z={\Phi }^{-1}(1-\delta )$. The central limit theorem without continuity correction gives $n-MN\ge z\sqrt{n(M-1)}$. Solving this quadratic inequality in $\sqrt{n}$ gives the displayed threshold, rounded upward. It exceeds the average stopping cost $MN$ because it allows for fluctuations in the accepted count. For $M=1$, the count is deterministic and $n=N$ exactly. Exact coverage uses the binomial distribution tail rather than the approximation.

Conditional iid accepted values have the ordinary target-sample CLT at scale $\sqrt{N}$. Since $N/n\to 1/M$, the same estimator has asymptotic variance $M{\mathrm{Var}}_f\varphi$ at scale $\sqrt{n}$. The centered count has limiting variance $(1/M)(1-1/M)$. Any fixed convention on an empty output is asymptotically negligible.

With normalized densities $f,g$ and envelope $f\le Mg$, $M>1$, conditioning a proposal on rejection gives density $r=(Mg-f)/(M-1)$. It can be used as an importance sampling proposal only if it covers the relevant target support. If $Mg=f$ on a set of positive target mass, rejected draws never visit that set.

Adaptive rejection sampling builds a piecewise linear upper envelope for a differentiable log-concave probability density, samples from its piecewise exponential form, and accepts with the target-to-envelope ratio. Adding evaluated points improves the envelope without changing the target distribution. Initial tangent slopes must yield integrable tails. The accepted draws are exact samples, unlike a finite run of Markov chain Monte Carlo.

To estimate ${\mathbb{E}}_f[g(X)]$, sample $Y_1,\dots ,Y_m$ independently from a reference density $h$ whose support covers that of $gf$, and useThe summands have expectation $\int g(x)f(x)dx$, and the strong law of large numbers gives almost-sure convergence under integrability.

To estimate $\mathbb{P}(Z>z)$ for a standard normal variable using proposal $N(\mu ,1)$, the weighted indicator has second moment $e^{{\mu }^2}\bar{\Phi }(z+\mu )$. Differentiating it at a minimum gives the displayed equation. Since $\phi (a)/\bar{\Phi }(a)=E[Z\mid Z>a]$, the minimizing shift is half the target conditional mean above the shifted threshold. The resulting estimator remains unbiased with the exact likelihood-ratio weight.

For target $p=N(0,{\sigma }^2)$ and proposal $q=N(0,1)$, this is the importance weight. Assuming only $E_p{f}^2<\infty$, the weighted estimator has guaranteed finite variance for $0<\sigma \le 1$, since $E_q(w_{\sigma }^2{f}^2)=E_p(w_{\sigma }{f}^2)\le {\sigma }^{-1}{E}_p{f}^2$. For every $\sigma >1$, choosing $f^2=e^{a{x}^2}$ with $\max (0,{\sigma }^{-2}-1/2)<a<(2{\sigma }^2{)}^{-1}$ gives finite target second moment but infinite weighted second moment. With the additional assumption that $f$ is bounded, the sufficient range is instead $\sigma <\sqrt{2}$. The integrand hypothesis therefore matters.

Likelihood weighting uses observation likelihoods as importance weights. If parameters are drawn independently from their prior distribution $\rho$, weight a draw by $L(\theta )$ and estimate a posterior expectation of $h$ by ${\sum }_{j}L({\theta }_j)h({\theta }_j)/{\sum }_{j}L({\theta }_j)$. With positive finite evidence and integrable weighted quantities, the strong law of large numbers proves consistency of this self-normalized importance sampling ratio. Finite-sample bias and concentration of the weights remain possible. A different proposal requires the additional prior density-to-proposal density ratio.

Let $r$ be an unnormalized density, $Z=\int r$ with $0<Z<\infty$, and $\pi =r/Z$. Given independent draws ${\theta }_t\sim \pi$ and an evaluable normalized density $q$ supported where $r>0$, reciprocal importance sampling estimates $Z^{-1}$ by $T^{-1}{\sum }_{t}q({\theta }_t)/r({\theta }_t)$. Its expectation is $Z^{-1}\int q=Z^{-1}$ and its variance is $(T{Z}^2{)}^{-1}(\int q^2/\pi -1)$. Broad tails of $q$ relative to $\pi$ can make this infinite; reversing the importance sampling ratio reverses the dangerous tail mismatch. Taking the reciprocal of the estimate is generally biased for $Z$.

For $U$ uniform on $(0,1)$, $Y$ has density $q(y)=2y^{-2}$ on $y\ge 2$. The importance sampling weight for the standard Cauchy distribution tail is $W=f(Y)/q(Y)$, and its expectation is the target tail probability. Writing $A=\arctan (1/2)$ gives $\mathbb{E}W=A/\pi$ and $\mathbb{E}{W}^2=(A+2/5)/(4{\pi }^2)$ by integrating $y^2/(1+y^2{)}^2$. The weight varies only between $2/(5\pi )$ and $1/(2\pi )$, making its variance much smaller than that of a rare-event indicator. This is a comparison of sampling variances, not a deterministic ordering of every pair of empirical variance estimates.

For rejection indicators with conditional acceptance $f(X_i)/(Mg(X_i))$, the conditional expectation of $(M/n){\sum }_i{I}_i\varphi (X_i)$ given all proposals is the unnormalized importance estimator. The Rao-Blackwell theorem reduces variance relative to that fixed-denominator estimator. It gives no such automatic comparison with the accepted-sample mean, whose denominator is the random count.

If $f\le Mg$, the importance weight $w=f/g$ is bounded by $M$, and ${\mathbb{E}}_g[{\varphi }^2{w}^2]\le M{\mathbb{E}}_f{\varphi }^2$. A finite target second moment therefore guarantees finite variance and the iid central limit theorem for the unnormalized importance-sampling average. Its proposal-budget variance is ${\mathbb{E}}_g[{\varphi }^2{w}^2]-({\mathbb{E}}_f\varphi {)}^2$.

For $\mu =\int hf$ and $0<A=\int |h|f<\infty$, the proposal $q_{\ast }=|h|f/A$ minimizes the variance of the ordinary unbiased importance sampling estimator. The Cauchy-Schwarz inequality gives $\int h^2{f}^2/q\ge A^2$, so the minimum variance for $n$ independent samples is $(A^2-{\mu }^2)/n$. The proposal need only cover the nonzero integrand.

An importance sampling proposal $q$ must be positive wherever the target-weighted integrand can contribute. The usual sufficient condition is that the target probability is absolutely continuous with respect to $q$. Otherwise multiplying observed draws by $f/q$ cannot recover a missing part of the integral. A strict rejection envelope supplies this condition for the rejected-proposal distribution.

For target density $f$ and proposal density $h$, the importance weight is the density ratio $w(x)=f(x)/h(x)$ wherever the target is supported.

For an unnormalized target density, self-normalized importance sampling uses weights $w_i=f(Y_i)/h(Y_i)$ and estimates expectations by ${\sum }_i{w}_{i}g(Y_i)/{\sum }_i{w}_i$. Resampling a point $Y_i$ with probability proportional to $w_i$ produces the associated weighted empirical distribution.

For estimating $I=\int g(x)f(x)dx$ by the unbiased importance estimator $g(Y)f(Y)/q(Y)$, the proposal minimizing variance isThis follows from the Cauchy-Schwarz inequality. Its normalizing constant and dependence on the particular integrand often make it impractical.

For normalized importance weights $w_i$, the standard effective sample size is $K_{\mathrm{eff}}=1/{\sum }_i{w}_i^2$.