Past exam of the mathematics course of the University of Cambridge / 2005 / ii / Paper 4 / 13I (Statistical Modelling)

The density is that of a Gamma distribution with shape $\nu >0$ and rate $\nu /{\mu }_i$, requiring ${\mu }_i>0$. Direct integration givesfor $\nu +r>0$. Using $\Gamma (\nu +1)=\nu \Gamma (\nu )$ and $\Gamma (\nu +2)=\nu (\nu +1)\Gamma (\nu )$,

Put $t_i={\beta }^T{x}_i=1/{\mu }_i>0$. Up to constants independent of $\beta$, the log-likelihood isThe score function and its derivative areThus an interior maximum-likelihood estimator solvesFor a full-rank design the Hessian is negative definite on the parameter region, so there is at most one interior solution and it is the maximum. A Newton method, also Fisher scoring here because the observed Hessian equals its expectation, isRecompute the means at every step and use a line search if necessary to keep all $t_i>0$ and increase the log-likelihood. Existence, identifiability and nonsingularity are regularity conditions, not consequences of the density alone.

The Fisher information isUnder standard interior, full-rank and large-sample regularity conditions, the maximum-likelihood estimator is asymptotically normal with mean $\beta$ and covarianceThereforeThe sums already depend on sample size; no extra factor $1/n$ should be inserted. The variance is estimable by substituting fitted means. If all $z_i$ are the same, $ac-b^2=0$ and the two coefficients are not separately identifiable.