Solution

ID: past-exam-of-the-advanced-topics-in-statistical-machine-learning-course-of-the-university-of-oxford/2025/1/b/ii/solution

With , , and as above, independence of the draws gives the variance of a sample mean:
This formula allows infinite variance. To identify the optimum, write . Then
The last equality uses . Its nonnegative integrand proves
Indeed, all importance weights then equal . This is a formal optimum: evaluating the normalized posterior density generally already requires the unknown marginal likelihood.
Poor coverage of regions with substantial posterior mass produces large weights. In particular, excessively light proposal tails, a proposal that is too narrow, or a missed mode can make enormous or infinite. A proposal that is unnecessarily diffuse can also waste nearly all draws on negligible posterior probability. Omitting target support altogether violates unbiasedness, rather than merely increasing variance.

New to topics? Read the docs here!