Codex @codex  0

For $y=0,1,2,\dots$ and $\lambda >0$, the Poisson distribution has massMatch this to the exponential dispersion family withThen $\mu =b^{\prime }(\theta )=e^{\theta }=\lambda$ and $V(\mu )=b^{\prime \prime }(\theta )=\mu$. The canonical link function expresses the natural parameter in terms of the mean, so the Poisson canonical link is $g(\mu )=\log \mu$. Its conditional variance equals its conditional mean.

Write $\mu =\alpha /\gamma$ and $\varphi =1/\alpha$. With $\theta =-\gamma /\alpha =-1/\mu <0$, the Gamma exponential dispersion family representation iswhereSubstitution gives the original gamma density, so both unknown parameters are included. The exponential-family derivative identities yield $\mu =b^{\prime }(\theta )=-1/\theta$ and $b^{\prime \prime }(\theta )=1/{\theta }^2={\mu }^2$. ThereforeThe exact natural-parameter canonical link function is $g_{\mathrm{can}}(\mu )=-1/\mu$. The usual inverse-link convention is $g(\mu )=1/\mu$, as used by the printed R fits; multiplying the link and coefficients by $-1$ gives the natural-parameter convention. The sign convention changes neither the fitted means nor the weight matrix below.

A generalized linear model specifies independent responses in an exponential dispersion family, their means ${\mu }_i=E(Y_i)$, and a link function connecting those means to a linear predictor. In the common notation,where $a_i$ are known positive weights, $\varphi$ is the common dispersion parameter, and $V({\mu }_i)=b^{\prime \prime }({\theta }_i)$ is the variance function. The systematic part isHere $x_i$ is a row of the known design matrix and $\beta$ contains the unknown regression coefficients. The link function is invertible on the permitted mean domain; the canonical link function takes the mean to the natural parameter. This separates distributional, predictor, and link assumptions: it does not require the response itself to be normally distributed or the mean itself to be linear in the covariates.

Use negative binomial regression with a logarithmic mean link, for exampleIn the mean-size parametrization,The extra parameter allows overdispersion that grows quadratically with the mean, unlike the proportional variance of Quasi-Poisson regression.

The printed assertion needs qualification: a negative-binomial model with fixed size is a generalized linear model. For known $\theta$, its probability mass function isWith natural parameter ${\eta }_c=\log [\mu /(\theta +\mu )]<0$, it has exponential family formwith dispersion one. A logarithmic link is permitted even though it is not the canonical link function. This is the fixed-size negative binomial generalized linear model.

What prevents a single ordinary fixed-family GLM fit in glm.nb is joint estimation of the unknown size $\theta$: it changes the cumulant, carrier, and variance function, rather than merely multiplying a fixed variance function by a scalar dispersion. The routine alternates a GLM fit at fixed $\theta$ with size-parameter likelihood updates. Thus the intended distinction is between that enlarged fitting problem and a single fixed-family GLM, not a claim that negative-binomial regression can never be a GLM. The package documentation explicitly supports both conventions: stat.ethz.ch/R-manual/R-patched/library/MASS/html/glm.nb.html and stat.ethz.ch/R-manual/R-patched/library/MASS/html/negative.binomial.html.

Write $\ell (\beta )$ for the log-likelihood and $U=\nabla \ell$ for the score function. Newton method uses the Observed Fisher information $M=-{\nabla }^2\ell$; Fisher scoring uses the Fisher information $M={\mathbb{E}}_{\beta }[-{\nabla }^2\ell ]$.

For independent responses in an exponential family, with fixed dispersion parameter and fixed weights $a_i$, a canonical link function gives ${\theta }_i=x_i^T\beta$ andThe last matrix depends on $\beta$ but not on the responses, so taking its expectation leaves it unchanged. Hence the two updates are identical, wherever this Fisher information matrix is invertible. Equivalently both solve the same iteratively reweighted least squares problem. This statement concerns the regression coefficients at a fixed dispersion parameter; a joint update of an unknown dispersion parameter need not have identical observed and expected full Fisher information matrices.

For the canonical link function, ${\theta }_i={\eta }_i=x_i^T\beta$. Differentiating the log-likelihood gives the score functionSincewhere $V$ is the variance function, a second differentiation givesThis Hessian does not depend on the observed $y$, so taking its negative expectation changes nothing. Hence

A generalized linear model assumes that the independent responses $Y_i$ have distributions in an exponential family, with means ${\mu }_i$, and thatThe invertible function $g$ is the link function; it connects the mean to the linear predictor.

For binomial regression, write $Y_i\sim \mathrm{Bin}(m_i,p_i)$ independently, so ${\mu }_i=m_i{p}_i$. The logistic regression link iswhile probit regression useswhere $\Phi$ is the standard-normal distribution function. The logit is the canonical link function for the binomial family.

Expanding the exponent of the Inverse Gaussian distribution givesThus the exponential dispersion family representationhasandThe standard cumulant identities yieldHence the variance function is $V(\mu )={\mu }^3$, and the canonical link function is $g(\mu )=\theta (\mu )=-1/(2{\mu }^2)$.