Past exam of the mathematics course of the University of Cambridge / 2004 / ib / Paper 4 / 19H (Statistics)

A Bayes estimator minimizes the integrated loss function under the prior and sampling distribution. For a nonnegative loss, Tonelli's theorem and the posterior distribution give the risk decompositionwhere $m$ is the marginal density. Thus choosing, for each observed $x$, a measurable minimizer of the posterior expected loss minimizes the integrated risk. Observations of marginal probability zero do not affect the rule.

Here the likelihood is $\theta e^{-\theta x}$ and the prior is $\mu e^{-\mu \theta }$ on positive parameters. Put $b=\mu +x$. Their product is $\mu \theta e^{-b\theta }$, and ${\int }_0^{\infty }\theta e^{-b\theta }d\theta =b^{-2}$. The marginal density and posterior are thereforeThis Gamma-exponential conjugacy calculation gives a Gamma distribution with shape $2$ and rate $b$. It is the posterior, rather than the prior alone, which determines both requested estimators.

Under squared-error loss, the posterior expected loss decomposes asIt is uniquely minimized by the posterior mean. Integrating the Gamma posterior gives $\mathbb{E}[\theta \mid x]=b^2{\int }_0^{\infty }{\theta }^2{e}^{-b\theta }d\theta =2/b$. Hence the Bayes estimator is

This relative absolute-error loss weights each parameter by $1/\theta$. Its posterior expected loss isThe effective density for the Bayes estimator under weighted absolute loss is proportional to $\pi (\theta \mid x)/\theta$, hence an exponential distribution of rate $b$. Its median solves $1-e^{-ba}=1/2$.

More explicitly, for $a\ge 0$, differentiating the split integrals at $\theta =a$ gives $R_x^{\prime }(a)=b(1-2e^{-ba})$ and $R_x^{\prime \prime }(a)=2b^2{e}^{-ba}>0$. For $a<0$, the loss has derivative $-b$, so its minimum cannot lie there. The unique global minimum is thereforeThis is the weighted posterior median; the ordinary median of the Gamma posterior would minimize unweighted absolute loss and would give a different answer.