Past exam of the mathematics course of the University of Cambridge / 2020 / ii / Paper 1 / 5J (Statistical Modelling)

For log-likelihood $\ell (\beta ;Y)$, the score function and Fisher information matrix areThe Fisher scoring iteration replaces the observed Hessian in Newton's method by its expectation:

The weighted least squares objective isIts gradient isSince $W$ is positive diagonal and $X$ has full column rank, $X^{T}WX$ is positive definite. The unique stationary point is therefore the unique minimizer:

At a current value $\beta$, putand define the working responseThen the displayed formulas in the question can be written compactly asandIndeed, the $j$th component of the latter iswhich is precisely the stated score.

Substitution into the Fisher-scoring update givesBy part (b), this is the weighted least squares fit of the working response $z$ on $X$. Both $z$ and $W$ are recomputed from the current fitted means after every update. Fisher scoring for a generalized linear model is therefore the iteratively reweighted least squares algorithm.