Past exam of the mathematics course of the University of Cambridge / 2004 / iii / Paper 37 / 1

Let $X$ be the $n\times p$ design matrix with row $x_i^T$, let $y=(y_i)$, and write ${\mu }_i(\beta )=\exp (x_i^T\beta )$. Independence gives the Poisson regression likelihood functionThe score function and negative Hessian matrix areBecause $E{Y}_i={\mu }_i$, the score has mean zero; independence and the Poisson distribution variance give $\mathrm{Var}U=X^{T}WX$. Thus the observed and expected Fisher information matrices are the same here.

Assume $X$ has full column matrix rank $p$. For a nonzero vector $a$, $a^{T}Ia={\sum }_i{\mu }_i(x_i^{T}a{)}^2>0$. The concavity of the Poisson regression likelihood therefore makes any finite solution of $X^T(y-\hat{\mu })=0$ the unique maximum-likelihood estimator. If the design is rank deficient, parameters differing by a vector in $\ker X$ give the same distribution, so only identifiable combinations can be estimated. Full rank alone does not ensure a finite maximizer: for an intercept-only model with all observations zero, $\ell =-n{e}^{\beta }$ approaches its supremum as $\beta \to -\infty$. More generally, directions that decrease only fitted means attached to zero counts can produce boundary fits. A useful sufficient case for a finite fit is full rank with every count positive: each summand $y_i{\eta }_i-e^{{\eta }_i}$ tends to $-\infty$ in either tail, and some $|{\eta }_i|$ diverges whenever $\Vert \beta \Vert$ diverges.

For an interior fit, Newton's method and Fisher scoring give the same update:This is also iteratively reweighted least squares. Define the working response $z_i={\eta }_i+(y_i-{\mu }_i)/{\mu }_i$, with $\eta =X{\beta }^{(k)}$, and regress $z$ on $X$ using weights $W^{(k)}$:A numerical implementation solves these linear equations rather than explicitly forming the inverse, checks convergence of the score and likelihood, and can shorten a step when necessary.

Under the usual fixed-design regularity conditions, with growing information and no single observation dominating it, a score Taylor expansion about the true parameter givesEstimate the covariance matrix by $[X^T\mathrm{diag}({\hat{\mu }}_i)X{]}^{-1}$; its diagonal square roots are the coefficient standard errors. This justifies large-sample Wald confidence intervals and Wald tests for specified coefficients or contrasts. Finite-sample unbiasedness is not being asserted. A likelihood-ratio test compares nested fits through twice their maximized log-likelihood difference. The Poisson deviance against the saturated fit iswith $0\log 0=0$. Residual diagnostics and a check of overdispersion assess whether the assumed Poisson variance is adequate; the variance correction required when it is not is derived in the next part.

The working score function and its derivative areAt an interior working maximizer, $U(\hat{\beta })=0$. A Taylor expansion at ${\beta }_0$ yieldsfor an intermediate value ${\beta }^{\ast }$. When the last term is negligible relative to the linear one, this givesThis is the Poisson score linearization under proportional variance. Correct specification of the mean alone implies $EU({\beta }_0)=0$. The stipulated independence and variance relation giveTaking the mean and variance of the linear approximation therefore provesThese are first-order conclusions, not exact finite-sample identities. The mean and variance assumptions alone do not guarantee that the approximation is accurate: an interior consistent solution, increasing information, control of leverage, and a suitable central limit condition are needed for normal inference.

The quasi-likelihood score for variance $\varphi {\mu }_i$ is $U/\varphi$, so its zero is unchanged by the unknown common dispersion parameter. Thus the working Poisson estimating equation remains appropriate for the mean parameter, but its unadjusted Poisson variance estimate is wrong when $\varphi \ne 1$. A usual Pearson dispersion estimator for this one-parameter model isIn particular overdispersion multiplies coefficient standard errors by approximately $\sqrt{\varphi }$, even though it leaves the coefficient estimating equation unchanged.