= Solution
Write $\ell(\beta)$ for the <log-likelihood> and $U=\nabla\ell$ for the <score function>. <Newton method> uses the <Observed Fisher information> $M=-\nabla^2\ell$; <Fisher scoring> uses the <Fisher information> $M=\mathbb E_\beta[-\nabla^2\ell]$.
For independent responses in an <exponential family>, with fixed <dispersion parameter> and fixed weights $a_i$, a <canonical link function> gives $\theta_i=x_i^T\beta$ and
$$
\ell(\beta)=\sum_i\frac{y_i\theta_i-b(\theta_i)}{a_i}+\sum_i c_i(y_i),\qquad
U=\sum_i\frac{x_i(y_i-b'(\theta_i))}{a_i},\qquad
-\nabla^2\ell=\sum_i\frac{b''(\theta_i)}{a_i}x_ix_i^T.
$$
The last <matrix> depends on $\beta$ but not on the responses, so taking its <expectation> leaves it unchanged. Hence \b[the two updates are identical], wherever this <Fisher information matrix> is invertible. Equivalently both solve the same <iteratively reweighted least squares> problem. This statement concerns the regression coefficients at a fixed <dispersion parameter>; a joint update of an unknown <dispersion parameter> need not have identical observed and expected full <Fisher information matrices>.
Back to article page
