Past exam of the options II course of the University of Oxford / 2011 / O (Statistics)

Ignoring a factor independent of the parameters, the likelihood function isSince every count is positive, a boundary value ${\theta }_{ij}=0$ has zero likelihood. At an interior optimum, a Lagrange multiplier for $\sum {\theta }_{ij}=1$ gives $n_{ij}/{\theta }_{ij}=\lambda$. Summing the resulting ${\theta }_{ij}=n_{ij}/\lambda$ gives $\lambda =N$, soThe log likelihood function is strictly concave on the positive simplex, because its Hessian matrix is diagonal with entries $-n_{ij}/{\theta }_{ij}^2<0$. Hence this critical point is the unique global maximum-likelihood estimate.

Under independence, the log likelihood function separates:Maximizing the two terms over their respective probability simplices, by the same Lagrange multiplier argument as in part (a), givesTherefore the constrained maximum-likelihood estimates areThe associated fitted cell counts are $E_{ij}=N{\hat{\theta }}_{ij}^0=n_{i.}{n}_{.j}/N$.

The log likelihood ratio statistic comparing the unrestricted fit with the independence fit isThe unrestricted model has $rc-1=8$ free parameters. Independence has $(r-1)+(c-1)=4$, so the difference is $(r-1)(c-1)=4$. By Wilks theorem, under the null hypothesis the statistic has asymptotically a chi-squared distribution with $4$ degrees of freedom, rather than $6$, $8$, or $9$.

For the supplied statistic, the approximate p-value is $1-F_4(11.8)=0.02$. Thus reject independence at the 5% level, but not at the 1% level: the data provide evidence of an association. The approximation is reasonable here; the smallest fitted cell count is $27\cdot 99/299\approx 8.94$, so all fitted counts exceed $5$. The statistical hypothesis test concerns association, not a causal effect of personality.

The Gaussian linear model has log likelihood function, up to an additive constant,For each fixed positive ${\sigma }^2$, maximize this by minimizing the residual sum of squares. Differentiation gives the normal equationsThe full column rank of $X$ makes $X^{T}X$ positive definite, hence invertible, soThe quadratic objective is strictly convex, which proves uniqueness. The joint maximum-likelihood estimate has the same coefficient estimate, except on the probability-zero event of an exactly zero residual when the positive-variance likelihood has no finite joint maximizer.

Write $A=(X^{T}X{)}^{-1}{X}^T$. Since $AX=I_p$,The errors have mean zero and covariance matrix ${\sigma }^2{I}_n$. ThereforeThus $\hat{\theta }$ is an unbiased estimator. In addition, the multivariate normal distribution is preserved by linear maps, so $\hat{\theta }$ is multivariate normal with the displayed mean and covariance matrix.

Use the residual mean squareTo justify its denominator, let $H=X(X^{T}X{)}^{-1}{X}^T$, the orthogonal projection matrix onto the column space of $X$. Then the residual is $(I-H)ϵ$. The idempotent matrix $I-H$ has rank $n-p$, so diagonalizing it with an orthogonal matrix givesConsequently $\mathbb{E}{s}^2={\sigma }^2$, making $s^2$ an unbiased estimator. If specifically asked for maximum likelihood estimation, maximizing over ${\sigma }^2$ gives the alternative ${\hat{\sigma }}_{\mathrm{ML}}^2=\Vert Y-X\hat{\theta }{\Vert }^2/n$, which is biased downward by the factor $(n-p)/n$.

Let $v_i=[(X^{T}X{)}^{-1}{]}_{ii}>0$. Part (b) givesThe residual vector and coefficient estimator are independent: they are jointly normal and their cross-covariance matrix is zero, since $X^T(I-H)=0$. Hence $Z_i$ is independent of $U=(n-p)s^2/{\sigma }^2\sim {\chi }_{n-p}^2$. By the definition of Student's t-distribution,Let $t_{n-p,1-\alpha /2}$ be the $1-\alpha /2$ quantile of this distribution. Inverting the central probability statement gives the exact marginal confidence intervalIts coverage is $1-\alpha$ under the stated Gaussian linear model; simultaneous coverage for all coefficients would require a separate adjustment.