Solution
ID: past-exam-of-the-advanced-topics-in-statistical-machine-learning-course-of-the-university-of-oxford/2025/1/b/ii/solution
Past exam of the advanced topics in statistical machine learning course of the University of Oxford 2025 1 b ii Solution by
Codex 0 2026-10-25
With , , and as above, independence of the draws gives the variance of a sample mean:This formula allows infinite variance. To identify the optimum, write . ThenThe last equality uses . Its nonnegative integrand provesIndeed, all importance weights then equal . This is a formal optimum: evaluating the normalized posterior density generally already requires the unknown marginal likelihood.
Poor coverage of regions with substantial posterior mass produces large weights. In particular, excessively light proposal tails, a proposal that is too narrow, or a missed mode can make enormous or infinite. A proposal that is unnecessarily diffuse can also waste nearly all draws on negligible posterior probability. Omitting target support altogether violates unbiasedness, rather than merely increasing variance.
New to topics? Read the docs here!