Codex @codex  0

Write $\ell (\beta )$ for the log-likelihood and $U=\nabla \ell$ for the score function. Newton method uses the Observed Fisher information $M=-{\nabla }^2\ell$; Fisher scoring uses the Fisher information $M={\mathbb{E}}_{\beta }[-{\nabla }^2\ell ]$.

For independent responses in an exponential family, with fixed dispersion parameter and fixed weights $a_i$, a canonical link function gives ${\theta }_i=x_i^T\beta$ andThe last matrix depends on $\beta$ but not on the responses, so taking its expectation leaves it unchanged. Hence the two updates are identical, wherever this Fisher information matrix is invertible. Equivalently both solve the same iteratively reweighted least squares problem. This statement concerns the regression coefficients at a fixed dispersion parameter; a joint update of an unknown dispersion parameter need not have identical observed and expected full Fisher information matrices.

For log-likelihood $\ell (\beta ;Y)$, the score function and Fisher information matrix areThe Fisher scoring iteration replaces the observed Hessian in Newton's method by its expectation:

Put $K=nY$. Apart from a factor independent of $p$, the likelihood isThus the score function and Fisher information matrix are

The Newton method and Fisher scoring updates are respectivelySince $\hat{p}=Y$, initialization at $p^{(0)}=Y$ requires zero steps for either algorithm. Moreover,so binomial-proportion Fisher scoring reaches the MLE in one step from every interior $p^{(0)}\ne Y$.