Past exam of the mathematics course of the University of Cambridge / 2019 / ii / Paper 3 / 28J (Principles of Statistics)

Let $S_n={\sum }_{i=1}^n{X}_i$. The log-likelihood iswith score function $n/\theta -S_n$ and second derivative $-n/{\theta }^2<0$. The unique maximum-likelihood estimator is thereforeFor an exponential distribution of rate $\theta$,The central limit theorem givesApplying the delta method to $h(x)=1/x$, for which $h^{\prime }(1/\theta )=-{\theta }^2$, yields the exponential-rate maximum-likelihood estimator limit

Multiplying the likelihood by the gamma distribution prior density givesThus Gamma-exponential conjugacy giveswhere the second parameter is the rate. Under squared-error loss, the Bayes estimator under squared error loss is the posterior mean, hence

The strong law of large numbers gives $S_n/n=\overline{X}\to 1/\theta$ almost surely, and hence in probability. Thereforeby the continuous mapping theorem. Thus the Bayes estimator has consistency.

It differs from the maximum-likelihood estimator bySince $S_n/n\to 1/\theta$, the numerator is $O_p(n)$ and the denominator is $O_p(n^2)$, so this difference is $O_p(n^{-1})=o_p(n^{-1/2})$. The limit in part a and Slutsky theorem now give