Past exam of the mathematics course of the University of Cambridge / 2008 / ii / Paper 4 / 13J / Solution

Write ${\theta }_i=\theta ({\mu }_i)$ and ${\eta }_i=x_i^T\beta =g({\mu }_i)$. Differentiating ${\mu }_i=K^{\prime }({\theta }_i)$ gives $d{\mu }_i/d{\theta }_i=V({\mu }_i)$, soIn the independent exponential dispersion family, only $[y_i{\theta }_i-K({\theta }_i)]/{\sigma }_i^2$ depends on $\beta$. Its derivative therefore gives the score functionThe residuals are independent, have mean zero and variance ${\sigma }_i^{2}V({\mu }_i)$. Consequently their score covariance is the Fisher information matrix:This also equals minus the expected likelihood Hessian under the usual regularity conditions.

For the canonical link function, $g(\mu )=\theta (\mu )$ and $g^{\prime }=1/V$, soTo obtain the maximum-likelihood estimate, solve $U(\hat{\beta })=0$, for example with Fisher scoring iterations ${\beta }^{new}=\beta +\mathcal{I}(\beta {)}^{-1}U(\beta )$. Equivalently use iteratively reweighted least squares, with weights $[{\sigma }_i^{2}V({\mu }_i)g^{\prime }({\mu }_i{)}^2{]}^{-1}$ and working response ${\eta }_i+(y_i-{\mu }_i)g^{\prime }({\mu }_i)$. Since ${\sigma }_i^2={\sigma }^2{a}_i$, the common dispersion scale cancels from this update for $\beta$. A nonsingular information matrix and a finite interior likelihood maximum are needed; iteration does not assert those for every possible dataset.