Past exam of the simulation and statistical programming course of the University of Oxford / 2023 / 2 / a

Write $w=p/q$ where $q>0$, assigning zero elsewhere. The envelope ensures $0\le w/M\le 1$, and $\pi$ is nonnegative. Integrating its joint probability density over the auxiliary coordinate givesConsequently ${\int }_{\Omega }{\int }_0^1\pi (x,u)dudx={\int }_{\Omega }p(x)dx=1$. This proves normalization of a probability distribution; the change of integration order is justified by Tonelli's theorem.

The coordinates are generally dependent. For $p(x)>0$, their conditional probability density isThus $U\mid X=x$ has a uniform distribution on an interval whose endpoint depends on $x$. They are independent exactly when $w(x)$ is constant for $p$-almost every $x$, since only then do these conditional laws agree. In particular, $p=q$ is an exception: $X\sim p$ and $U\sim \mathrm{Unif}(0,1/M)$ are independent. An unconditional answer that they can never be independent would therefore be too strong.

Integrating the joint probability density over the auxiliary coordinate, as in (i), gives the marginal distributionIn particular, augmenting with $U$ preserves the target probability distribution of $X$. The other marginal distribution, though not needed to identify $X$, is ${\pi }_U(u)=M{\int }_{\Omega }q(x)1_{\{w(x)\ge Mu\}}dx$ for $0<u<1$. It need not be a uniform distribution on $(0,1)$.

For the proposed joint probability density $\rho$, define the acceptance indicator functionOn the proposal's support the importance weight is $\pi (Y_i,V_i)/\rho (Y_i,V_i)=M{A}_i$. Therefore the requested augmented-space importance sampling estimator isIts expected value is $\theta$, provided ${\mathbb{E}}_p|\varphi (X)|<\infty$. This is not the ordinary estimator $n^{-1}{\sum }_{i}w(Y_i)\varphi (Y_i)$ from question 1. Taking the conditional expectation of this augmented estimator given all $Y_i$ produces that ordinary estimator, because $\mathbb{E}[A_i\mid Y_i]=w(Y_i)/M$. This is the Rao-Blackwell identity for a fixed-denominator rejection estimator.

Use the same acceptance indicator functions and set $K_n={\sum }_{i=1}^n{A}_i$, $S_n={\sum }_{i=1}^n{A}_i\varphi (Y_i)$. The rejection sampling estimator based on these $n$ proposals is the mean over the accepted observations:For $K_n=0$, any fixed convention, such as zero, is harmless asymptotically. The $A_i$ are independent and identically distributed random variables with Bernoulli distribution and success probability $1/M$, so the strong law of large numbers gives $K_n/n\to 1/M$ almost surely. In particular $K_n>0$ eventually almost surely. The exact identitythen holds whenever $K_n>0$. Wherever $S_n\ne 0$, it follows thatA qualification is missing from the printed claim: the quotient must be defined eventually. A convenient sufficient assumption is ${\mathbb{E}}_p|\varphi |<\infty$ and $\theta \ne 0$. Indeed, another application of the strong law of large numbers gives $S_n/n\to \theta /M$, ensuring that $S_n$ is eventually nonzero. This proves the stated quotient limit under that assumption.

As printed, the assertion is false as a universally defined quotient. For example, $\varphi \equiv 0$ makes both estimates zero for every $n$. Even excluding that example is insufficient: take $p=q$ to be the uniform distribution on $(0,1)$, $M=1$, and $\varphi (y)=1$ for $y<1/2$ and $-1$ otherwise. Then $S_n$ is a simple symmetric random walk, which returns to zero infinitely often almost surely, leaving the quotient undefined infinitely often.

With integrability but no assumption $\theta \ne 0$, the exact multiplicative identity remains valid, and both estimates converge to $\theta$ almost surely. In particular ${\hat{\theta }}_n^{\mathrm{IS}}-{\hat{\theta }}_n^{\mathrm{RS}}\to 0$ almost surely. One may alternatively define the quotient's removable $0/0$ cases as $M{K}_n/n$; that explicitly extended quantity always converges to one. These are valid repairs of the source's unrestricted statement.