Past exam of the mathematics course of the University of Cambridge / 2019 / ib / Paper 4 / 19H (Statistics)

Writeand assume $S_{xx}>0$ so that $\beta$ is identifiable.

Apart from terms independent of $\beta$, the Gaussian log-likelihood isDifferentiating with respect to $\beta$ givesThe unique zero, and hence the maximum-likelihood estimate, isThe same value minimizes the residual sum of squares ${\sum }_i(Y_i-\beta x_i{)}^2$, since maximizing the Gaussian likelihood for fixed ${\sigma }^2$ is equivalent to minimizing this sum. It is therefore also the least-squares estimator.

For this model, the Gauss-Markov theorem states that $\hat{\beta }$ is the unique minimum-variance linear unbiased estimator of $\beta$: ifis unbiased for every $\beta$, then $\mathrm{var}(\tilde{\beta })\ge \mathrm{var}(\hat{\beta })$.

Indeed,so unbiasedness is equivalent to ${\sum }_i{a}_i{x}_i=1$. WriteThe constraint becomes ${\sum }_i{c}_i{x}_i=0$. Since the errors are independent with common variance ${\sigma }^2$,Equality holds exactly when every $c_i=0$, which gives $a_i=x_i/S_{xx}$ and $\tilde{\beta }=\hat{\beta }$.

Any unbiased linear estimator has the form considered in part (b), and its error is the linear combination of independent normal random variablesThe moment-generating function of a normal distribution therefore gives, for every nonzero real $\theta$,Since the exponential function is strictly increasing and ${\theta }^2{\sigma }^2/2>0$, minimizing this exponential moment is exactly the same as minimizing ${\sum }_i{a}_i^2$. Part (b) shows that the unique minimizer is $a_i=x_i/S_{xx}$. Thus, independently of the sign or magnitude of $\theta$,