Past exam of the mathematics course of the University of Cambridge / 2006 / iii / Paper 47 / 5 / a / ii / Solution

An independence sampler proposes from a fixed density $g({\theta }^{\prime })$, independent of the current state. Its acceptance probability isIf $g$ is close to the target, proposals can make large efficient moves and cross separated modes; $g=\pi$ gives acceptance one and independent samples. On the other hand, a good global approximation is required. A proposal with tails too light can leave the chain trapped at states having very large importance weight $\pi /g$, since most outgoing proposals then have tiny acceptance probabilities.

For perspective, if the normalized target satisfies $\pi \le Mg$, the accepted transition density is at least $\pi ({\theta }^{\prime })/M$, because both entries in $\min \{g({\theta }^{\prime }),\pi ({\theta }^{\prime })/w(\theta )\}$ have that lower bound, where $w=\pi /g\le M$. This yields a uniform refresh component. Thus a well-designed independence proposal can mix rapidly, whereas a poor global proposal can be much worse than a locally tuned random walk. The tradeoff concerns effective samples per computation, not acceptance rate alone.