Past exam of the Probability, statistics and options II course of the University of Oxford / 2011 / O (Statistics)

Ignoring the multinomial coefficient, the likelihood is $L(\theta )={\prod }_{i,j}{\theta }_{ij}^{n_{ij}}$. Maximize $\ell ={\sum }_{i,j}{n}_{ij}\log {\theta }_{ij}$ with $\sum {\theta }_{ij}=1$. A Lagrange multiplier gives $n_{ij}/{\theta }_{ij}=\lambda$; summing the resulting ${\theta }_{ij}=n_{ij}/\lambda$ gives $\lambda =N$. ThusBecause every count is positive, the log-likelihood is strictly concave on the interior simplex and this solution is its unique maximum.

Under the product constraint the log-likelihood separates:Maximizing the two simplex problems independently gives ${\hat{\alpha }}_i=n_{i.}/N$ and ${\hat{\beta }}_j=n_{.j}/N$. HenceThe fitted expected cell counts under independence are consequently $N{\hat{\theta }}_{ij}^0=n_{i.}{n}_{.j}/N$.

The generalized likelihood ratio compares the constrained maximum from part (b) with the unrestricted maximum from part (a). Taking minus twice its logarithm givesThe unrestricted $3\times 3$ model has $9-1=8$ free parameters; the independence model has $(3-1)+(3-1)=4$. Thus the asymptotic null distribution is ${\chi }_4^2$, with four degrees of freedom. The supplied value givesTherefore reject independence at the 5% level, but not at the 1% level. There is evidence of an association between the two classifications. The fitted expected counts are all about nine or larger, so the usual chi-squared approximation has no very small expected cells here.

The normal-error log-likelihood, for fixed variance, is a constant minus $\Vert Y-X\theta {\Vert }^2/(2{\sigma }^2)$. Differentiating the squared residual gives the normal equations $X^{T}X\hat{\theta }=X^{T}Y$. Since $X$ has full column rank, $X^{T}X$ is positive definite and the squared residual is strictly convex in $\theta$. Hence the unique minimizer, and thus the maximum-likelihood estimator, isProfiling over the unknown positive variance does not change the residual-minimizing value of $\theta$.

Write $A=(X^{T}X{)}^{-1}{X}^T$. Since $AX=I$, the estimator is $\hat{\theta }=\theta +Aϵ$. ThereforeIts expectation proves unbiasedness. It is also jointly normal, being an affine transformation of the normal error vector.

Let $\mathrm{RSS}=\Vert Y-X\hat{\theta }{\Vert }^2$. The usual unbiased error-variance estimator isIndeed the residual is the orthogonal projection of $ϵ$ onto the $(n-p)$-dimensional complement of the design column space, giving $\mathrm{RSS}/{\sigma }^2\sim {\chi }_{n-p}^2$ and expected RSS equal to $(n-p){\sigma }^2$. The maximum-likelihood estimator instead divides RSS by $n$; the unbiased divisor above is the one used for the exact t-interval in part (d).

Put $C=(X^{T}X{)}^{-1}$ and $\nu =n-p$. The normal law of the estimator gives $({\hat{\theta }}_i-{\theta }_i)/(\sigma \sqrt{C_{ii}})\sim N(0,1)$. The fitted and residual Gaussian projections are orthogonal and hence independent, so this quantity is independent of $\nu s^2/{\sigma }^2\sim {\chi }_{\nu }^2$. Their ratio therefore has Student's t-distribution:If $t_{\nu ,1-\alpha /2}$ is its upper $1-\alpha /2$ quantile, rearranging its central probability event givesThis interval has exact coverage $1-\alpha$ under the given normal linear model, rather than merely large-sample coverage.