Past exam of the options II course of the University of Oxford / 2009 / O (Statistics)

Minimizing the sum of squared residuals gives the normal equations for linear least squares, $X^{T}X\hat{\theta }=X^{T}Y$. Full column rank makes $X^{T}X$ positive definite and invertible, so the unique minimizer isBecause $\mathbb{E}Y=X\theta$ and $\mathrm{Cov}(Y)={\sigma }^2{I}_n$, one has $AX=I_p$ andFor the alternative linear estimator the corresponding general formulas areNo normality assumption on the errors is needed for these claims.

Unbiasedness of an estimator means unbiasedness for every parameter value, not just one chosen $\theta$. Thus $DX\theta =\theta$ for every $\theta \in {\mathbb{R}}^p$. Evaluating this identity on a basis, or identifying the two linear maps, gives

Put $D^{\ast }=D-A$, with $A=(X^{T}X{)}^{-1}{X}^T$. Then $D^{\ast }X=0$, and therefore $D^{\ast }{A}^T=D^{\ast }X(X^{T}X{)}^{-1}=0$; its transpose also vanishes. Expanding givesHenceThe difference from the least-squares covariance is positive semidefinite, since $v^T{D}^{\ast }{D}^{\ast T}v=\Vert D^{\ast T}v{\Vert }^2\ge 0$. In particular its $i$th diagonal entry is ${\sigma }^2{\sum }_j(D_{ij}^{\ast }{)}^2$, proving every stated variance inequality. For estimating ${\theta }_1$, prefer least squares: among linear unbiased estimators it has minimum variance, and therefore minimum mean squared error in that class. A competitor can tie if the first row of $D^{\ast }$ is zero. This is the Gauss-Markov theorem; it does not assert optimality against all biased or nonlinear estimators.

For unrestricted category probabilities, the log-likelihood is ${\sum }_i{n}_i\log {\pi }_i$ subject to ${\sum }_i{\pi }_i=1$. A multiplier gives $n_i/{\pi }_i$ constant, hence the unique maximum, because all counts are positive and the log-likelihood is strictly concave:The likelihood ratio test compares this saturated maximum with the maximum under the parametric model. Twice the log ratio isFor $k$ bins and one regular fitted parameter, its usual large-sample reference distribution is ${\chi }_{k-2}^2$; reject for a sufficiently large value. Under a fully specified parameter ${\theta }_0$, use $\Lambda ({\theta }_0)$ and ${\chi }_{k-1}^2$. This relies on an identifiable interior fit and adequate expected bin counts.

For individual ungrouped abundances, let $S$ be the number of species and $T$ the total number of individuals. The logarithmic-series model givesIts score vanishes whenThe left side increases continuously from one to infinity. One way to see strict increase is to set $t=-\log (1-\theta )>0$, making it $(e^t-1)/t$, whose derivative is positive because $e^t(t-1)+1$ increases from zero. The score has the sign of $T/S$ minus this increasing function, so this is the unique maximum. Bisection on $(0,1)$ is a robust numerical method; Newton's method is another option.

There are two fits implicit in the supplied presentation. Using the ungrouped sufficient statistics gives ${\hat{\theta }}_{\mathrm{full}}\approx 0.944228$. By contrast the displayed ten-bin deviance is minimized near $0.93$, so that is the table-based estimate ${\hat{\theta }}_{\mathrm{grouped}}$. Grouping the upper tail discards its detailed abundance total; minimizing the grouped deviance maximizes the grouped likelihood, not the ungrouped likelihood used in the preceding score equation. The estimates therefore need not agree. The following table-based decisions use its grouped-fit convention.

The displayed table has ten categories, including one pooled upper-tail category. With the parameter fixed, no model parameter is fitted and the appropriate reference degrees of freedom are $10-1=9$. The supplied statistic is $15.59$, below the stated five-percent critical value $16.92$. Hence do not reject the specified log-series model at the five-percent level. Using eight degrees of freedom here would incorrectly charge a fitted parameter to a fully specified null.

For the one-parameter fit to ten grouped categories, the usual reference degrees of freedom are $10-1-1=8$. The table minimum is about $8.29$, far below the supplied critical value $15.51$, so do not reject the fitted log-series model at the five-percent level.

This standard grouped likelihood-ratio calibration uses the grouped maximum, approximately $0.93$, as the preceding answer explains. If the ungrouped estimate from the total abundance is inserted into a grouped statistic instead, the usual automatic loss of exactly one chi-squared degree of freedom is not justified by that same grouped-likelihood argument. The problem's table-based decision should therefore be kept distinct from that alternative procedure.