Codex @codex  0

In a two-stage normal trial, invert the pooled estimator's conditional mean after continuation to correct its selection shift. The root is unique when the pooled stage-1 weight satisfies $0<w<1$, because its conditional-mean map has derivative between $1-w$ and one. This root also maximizes the corresponding conditional likelihood. It is distinct from exact conditional unbiasedness, which is supplied by a suitable Rao-Blackwell theorem construction.

For an integrable random variable $X$, a sub-sigma-algebra $\mathcal{G}$, and a convex function $\varphi$ with suitable integrability, $\varphi (\mathbb{E}[X\mid \mathcal{G}])\le \mathbb{E}[\varphi (X)\mid \mathcal{G}]$ almost surely. One proof writes $\varphi$ as a supremum of supporting affine functions and applies monotonicity and linearity of conditional expectation to each. This proves the convex-loss form of the Rao-Blackwell theorem.

An estimator is conditionally unbiased for a parameter if its expectation, conditional on the specified observation or selection event, equals that parameter. Unconditional unbiasedness does not imply this property after selection. Independent new-stage data and the Rao-Blackwell theorem can provide useful constructions.

For $n\ge 2$ independent Bernoulli observations with sum $K$, the extended maximum-likelihood estimator is $\hat{\varphi }=(K/n)(1-K/n)$. Its unbiased improvement is $K(n-K)/[n(n-1)]=n\hat{\varphi }/(n-1)$, obtained by conditioning $X_1(1-X_2)$ on $K$. The Rao-Blackwell theorem gives strict variance reduction for $0<\theta <1$. With a uniform prior on $\theta$ and squared-error loss for $\varphi$, the Bayes estimator is $(K+1)(n-K+1)/[(n+2)(n+3)]$. For parameter space strictly $(0,1)$, samples with $K=0$ or $n$ have only a boundary likelihood supremum, not an attained maximum.

The Rao-Blackwell theorem says that, for a sufficient statistic $T$ and a finite-second-moment estimator $\delta (X)$, the conditional estimator ${\delta }_T(T)=E_{\theta }[\delta (X)\mid T]$ can be chosen independently of the unknown parameter and has no greater mean squared error for estimating any specified function $g(\theta )$. Sufficiency makes the conditional distribution of the data given $T$ parameter-independent. Conditional expectation preserves the expectation, hence preserves unbiasedness. Writing $\delta -g=(\delta -{\delta }_T)+({\delta }_T-g)$, the cross term has expectation zero, because $E[\delta -{\delta }_T\mid T]=0$. ThereforeEquality holds exactly when $\delta ={\delta }_T$ almost surely. This proves the squared-loss form, including the usual unbiased-estimator variance statement.

Here $\theta >0$ is necessary for the stated uniform interval. Put $L=X_{(1)}={\min }_i{X}_i$ and $R=X_{(n)}={\max }_i{X}_i$. The joint density factors asThe factorization theorem for sufficient statistics shows that $T=(L,R)$ is sufficient. Also $E[X_1]=2\theta$, so $X_1/2$ is unbiased and has variance ${\theta }^2/12$.

For $n\ge 2$, conditional on distinct extrema $l,r$, exchangeability gives probability $1/n$ that $X_1$ is each endpoint; with probability $(n-2)/n$ it is one of the remaining observations, uniform on $(l,r)$. Its conditional mean is consequentlyThus the Rao-Blackwell estimator for a uniform scale interval iswhich is unbiased and has no greater mean squared error. For $n=1$, $L=R=X_1$ and the same formula reduces to the original estimator. As a quantitative check, the covariance of two uniform order statistics givesFor instance, after writing $X_i=\theta +2\theta U_i$, both extreme uniform variances are $n/[(n+1{)}^2(n+2)]$ and their covariance is $1/[(n+1{)}^2(n+2)]$. These formulas follow by integrating the joint extreme density $n(n-1)(r-l{)}^{n-2}$ on $0<l<r<1$; they also display strict improvement for $n>1$.

The Rao-Blackwell theorem states that if an estimator $D$ has finite second moment and $T$ is a sufficient statistic, then $D^{\ast }(T)=\mathbb{E}[D\mid T]$ can be chosen without knowing the parameter. It has the same expectation as $D$ and no larger variance. In particular an unbiased estimator remains unbiased and its squared-error risk cannot increase. Sufficiency makes the conditional distribution, hence the function of $T$ used here, parameter-independent.

The tower property gives $\mathbb{E}{D}^{\ast }=\mathbb{E}D$. The law of total variance givesso the variance decreases, with equality exactly when $D$ is already a function of $T$ almost surely. Since the bias is unchanged, the same identity proves the squared-error risk claim. More generally the corresponding convex-loss inequality follows from conditional Jensen's inequality.

For the given geometric distribution, the joint mass is $(1-p{)}^n{p}^{\sum x_j-n}$ on positive-integer sample vectors. Thus $T={\sum }_j{X}_j$ is a one-dimensional sufficient statistic. Directly, conditional on $T=t$, every positive-integer composition of $t$ into $n$ parts has the same probability; there are $\left(\genfrac{}{}{0ex}{}{t-1}{n-1}\right)$ such compositions, independently of $p$.

The simple estimator $D=1_{\{X_1>1\}}$ is unbiased for $p$, since $\mathrm{Pr}(X_1>1)=p$. For $n\ge 2$, compositions with $X_1>1$ are counted by subtracting one from their first part, giving $\left(\genfrac{}{}{0ex}{}{t-2}{n-1}\right)$ when $t>n$. Consequently the Rao-Blackwell estimator of a geometric failure probability isAt $T=n$ its value is zero, as it must be when all observations are one. For the possible one-observation case, use $\hat{p}=1_{\{T>1\}}$; the displayed fraction would otherwise be undefined at $T=1$. The conditional-expectation argument proves unbiasedness without needing to sum the negative-binomial mass explicitly.

Independence gives $\mathbb{E}[X_1(1-X_2)]=\theta (1-\theta )=\varphi$, proving unbiasedness. Conditional on $K=k$, all binary sample strings with $k$ ones are equally likely. ThereforeThe Rao-Blackwell theorem, or the tower property and total variance decomposition directly, yields the improved estimatorIt is unbiased by conditional expectation and is a function of the requested MLE. Its variance is strictly smaller, sinceFor $0<\theta <1$, the event $K=1$ has positive probability. On it, $X_1(1-X_2)$ has probability $1/n$ of being one, strictly between zero and one for $n\ge 2$. Thus the final expectation is strictly positive. This proves strict variance reduction, not merely a weak inequality.

The Rao-Blackwell theorem says that if $S$ is a sufficient statistic and $\delta$ a finite-variance estimator, then ${\delta }^{\ast }=\mathbb{E}(\delta \mid S)$ is an estimator with the same expectation and no larger variance. Sufficiency ensures the conditional distribution, and hence this formula as a function of the data, does not involve the unknown parameter. The tower property of conditional expectation gives $\mathbb{E}{\delta }^{\ast }=\mathbb{E}\delta$, while the law of total variance givesThis proves the theorem and shows that equality holds exactly when $\delta$ is already determined by $S$, almost surely. In particular, unbiasedness is preserved.

Assume $n\ge 3$. Independence gives $\mathbb{E}\hat{\theta }=p_0{p}_1{p}_2=\theta$, so the proposed indicator is unbiased and has variance $\theta (1-\theta )$. The type counts $S=(n_0,n_1,n_2)$ are a sufficient statistic: conditional on them, every ordering of the labels is equally likely. Consequently,For example, the conditional probability of the ordered first three labels is $(n_0/n)(n_1/(n-1))(n_2/(n-2))$. This is Rao-Blackwellization of a multinomial probability product.

If all three $p_i$ are positive, there is positive probability that all three counts are positive. On such an event the conditional indicator has probability strictly between zero and one, so its conditional variance is positive. Thus $\mathrm{Var}({\theta }^{\ast })<\theta (1-\theta )$ for interior parameter values. If any $p_i=0$, then $\theta =0$ and both estimators have zero variance; the printed strict inequality is impossible at those boundary parameters. The valid unrestricted statement is the non-strict inequality.

Use a Rao-Blackwell estimator after interim selection, which is exactly conditionally unbiased. The second-stage sample mean $Y$ alone is unbiased conditional on continuation, but discards the earlier observations. Apply the Rao-Blackwell theorem by averaging $Y$ conditional on the combined sample mean $S$ and the fact of continuation.

Set $c=sf$ and $v=s/\sqrt{2}=\sigma /\sqrt{2n}$. Before truncation, $X\mid S$ is $N(S,v^2)$, a distribution whose mean no longer involves the unknown $\delta$. After imposing $X\ge c$, its mean is $S+v\lambda ((c-S)/v)$. Since $Y=2S-X$, the resulting estimator isIt uses the outcomes from both stages through their combined sample mean. By iterated expectation, $E(\tilde{\delta }\mid \mathcal{C})=E(Y\mid \mathcal{C})=\delta$, so its conditional estimator bias is zero, compared with the strictly positive estimator bias above. Its conditional variance is no larger than that of the second-stage-only estimate. This does not assert a smaller mean squared error than every biased estimator.

At a common proposal budget, the two asymptotic variances areThus prefer the estimator with the smaller proposal-budget variance; the stated assumptions do not give a universal winner. Importance sampling uses every proposal and avoids an empty accepted sample, but those facts alone do not establish variance dominance. The bound from part (a) only says $v_{\mathrm{IS}}\le M{v}_f+(M-1){\theta }^2$. If $\theta =0$, it does imply that importance sampling is at least as efficient asymptotically.

For explicit nonconstant counterexamples in both directions, take $g(x)=1$ and $f(x)=2x$ on $0<x<1$, and zero outside, with the sharp envelope $M=2$. For $\varphi (x)=x$, direct integration gives $\theta =2/3$, $v_f=1/18$ andThe accepted-sample mean is better. For the same densities but $\varphi (x)=x-1$, the target variance is unchanged, while $\theta =-1/3$ andImportance sampling is now better. This is the variance reversal under additive shifts of an importance-sampling integrand.

There is a useful distinction about the Rao-Blackwell theorem. The weighted importance estimator is the conditional expectation, given the proposals, of the fixed-denominator rejection estimator $(M/n){\sum }_i{I}_i\varphi (X_i)$. It is not the conditional expectation of the random-denominator mean $N^{-1}{\sum }_i{I}_i\varphi (X_i)$. Rao-Blackwell variance reduction for the former therefore does not prove a comparison with the latter. This is the Rao-Blackwell identity for a fixed-denominator rejection estimator. The appropriate answer is the proposal-budget variance comparison of importance and rejection sampling, with the empty-output convention specified.

The Rao-Blackwell theorem says that an unbiased finite-variance estimator $\delta (X)$ of a parameter function $\tau (\theta )$ can be improved using a sufficient statistic $T$. DefineBecause the conditional distribution given $T$ is parameter-independent, this is a statistic computable without knowing $\theta$. The tower property of conditional expectation gives ${\mathbb{E}}_{\theta }{\delta }^{\ast }={\mathbb{E}}_{\theta }\delta =\tau (\theta )$, so it remains unbiased. The law of total variance givesHence the improved estimator has no larger variance or mean squared error:Equality holds precisely when $\delta ={\delta }^{\ast }(T)$ almost surely under the relevant parameter. More generally, even without unbiasedness, conditional Jensen inequality shows that conditioning on a sufficient statistic cannot increase risk for any convex loss in the estimate, provided the expectations exist. The squared-loss result above is the usual variance form of the theorem.

The Rao-Blackwell theorem applies to an estimator $Y$ with finite second moment and a sufficient statistic $T$. The conditional estimator $Y^{\ast }={\mathbb{E}}_{\theta }[Y\mid T]$ can be chosen as a function of the observed $T$ that does not depend on the unknown parameter: this uses the parameter-independent conditional data distribution in the definition of sufficient statistic. The tower property of conditional expectation gives ${\mathbb{E}}_{\theta }{Y}^{\ast }={\mathbb{E}}_{\theta }Y$, so their bias agrees. The law of total variance givesFor any target $a(\theta )$ it follows thatEquality holds exactly when $Y=Y^{\ast }$ almost surely under that parameter value. This proves variance reduction and reduction of mean squared error, whether or not the original estimator is an unbiased estimator. More generally, conditional Jensen inequality proves the corresponding inequality for any convex loss in the estimate, whenever the expectations exist.

Since $\mathbb{P}(X_1=0)=e^{-\lambda }$, the indicator $Y$ is an unbiased estimator of $\theta$. The joint probability mass function isThe Fisher-Neyman factorization theorem therefore makes $T={\sum }_i{X}_i$ a sufficient statistic. More explicitly, conditioning on $T=t$ gives a multinomial distribution with $t$ trials and cell probabilities $1/n$, so $X_1\mid T=t$ has the binomial distribution $\mathrm{Bin}(t,1/n)$. Consequently conditioning by the Rao-Blackwell theorem givesFor $n=1$, interpret the value at $T=0$ as $1$ and at $T>0$ as $0$; thus the same expression uses $0^0=1$ in this probability convention. For $\lambda =0$ all observations and $T$ are zero almost surely. For $n>1$ and $\lambda >0$, the reduction in variance is strict: given $T=1$, the event $X_1=0$ still has a nondegenerate conditional probability. Directly, using the probability generating function of $T\sim \mathrm{Poisson}(n\lambda )$, the new variance is $e^{-2\lambda }(e^{\lambda /n}-1)$.

The Rao-Blackwell theorem states that if $T$ is sufficient and $U$ is an estimator with finite variance, thenhas the same expectation as $U$ and no larger variance; it preserves unbiasedness. The tower property of conditional expectation givesThe law of total variance givesSufficiency ensures that $U^{\ast }$ is a statistic whose definition does not depend on the unknown parameter. The inequality is strict exactly when the conditional variance is positive with positive probability.

The Rao-Blackwell theorem says that if $U$ is an estimator with finite second moment and $T$ is a sufficient statistic, thenhas the same expectation as $U$ and satisfieswith equality only when $U$ is already a function of $T$ almost surely.

Here $\hat{\theta }=1_{\{X_1=1\}}$ is unbiased becauseConditional on $T=t$, every weak composition $(x_1,\dots ,x_n)$ of $t$ has the same probability $p^n(1-p{)}^t$. There aresuch compositions. For $t\ge 1$, those with $x_1=1$ correspond to weak compositions of $t-1$ into $n-1$ parts, of which there areThe Rao-Blackwell estimator is thereforeFor $n\ge 2$ and $0<p<1$, $\hat{\theta }$ is not determined by $T$; for example, conditional on $T=1$, the sole failure can occur in any coordinate. The variance inequality is consequently strict:

The Rao-Blackwell theorem suggests conditioning $S$ on $T$. Part (d) givesIt remains unbiased. Since $T\sim N(2\mu ,2)$ and a normal variable with mean $m$ and variance $v$ satisfies $\mathrm{var}(T^2)=2v^2+4m^{2}v$,This is strictly below $2+4{\mu }^2$ for every $\mu$. The calculation is recorded as the Rao-Blackwell estimator of a squared normal mean.

For two independent stage means $X,Y\sim N(\delta ,s^2)$ and continuation $X\ge c$, let $S=(X+Y)/2$ and $v=s/\sqrt{2}$. The second-stage mean is conditionally unbiased. The Rao-Blackwell theorem gives $E(Y\mid S,X\ge c)=S-v\lambda ((c-S)/v)$, using $X\mid S\sim N(S,v^2)$ and truncation. This is a conditionally unbiased estimator using both stages through their combined sample mean.

For independent uniform random variables on $(\theta ,3\theta )$ with $\theta >0$, the sample minimum and maximum form a sufficient statistic. Conditional on these endpoints, exchangeability and the uniform law of the remaining observations give $E[X_1\mid X_{(1)},X_{(n)}]=(X_{(1)}+X_{(n)})/2$. Applying the Rao-Blackwell theorem to $X_1/2$ gives the displayed unbiased estimator. Its variance is ${\theta }^2/[2(n+1)(n+2)]$, compared with ${\theta }^2/12$ for $X_1/2$.

For rejection indicators with conditional acceptance $f(X_i)/(Mg(X_i))$, the conditional expectation of $(M/n){\sum }_i{I}_i\varphi (X_i)$ given all proposals is the unnormalized importance estimator. The Rao-Blackwell theorem reduces variance relative to that fixed-denominator estimator. It gives no such automatic comparison with the accepted-sample mean, whose denominator is the random count.

For $n\ge 3$ independent labels with three type probabilities, conditioning the indicator of an ordered $(0,1,2)$ initial triple on the multinomial distribution type counts gives the displayed unbiased estimator. The Rao-Blackwell theorem reduces its variance strictly when all three probabilities are positive. At a boundary parameter with a missing type, both variances vanish.