Fixed-size negative binomial generalized linear model ($\mathrm{Var}(Y)=\mu +{\mu }^2/\theta$)

For known size $\theta >0$, the negative binomial distribution is an exponential family with natural parameter $\eta =\log [\mu /(\theta +\mu )]<0$ and cumulant $b_{\theta }(\eta )=-\theta \log (1-e^{\eta })$. A generalized linear model can use a logarithmic link function for its mean, even though that link is not canonical. Jointly estimating the size changes the family and variance function, requiring additional updates beyond a single fixed-family GLM fit; this is what MASS::glm.nb does.

With known size $\theta >0$, the mean-dependent log mass of a negative binomial distribution is $y\log \mu -(y+\theta )\log (\mu +\theta )$. Its derivative is $\theta (y-\mu )/[\mu (\mu +\theta )]$, so the saturated mean is $y$, including its boundary value zero. Subtracting the fitted log mass from that maximum gives the displayed deviance. This formula needs no fitted-intercept cancellation. As $\theta \to \infty$, its second term tends to $-2(y_i-e_i)$ and it becomes the full Poisson deviance.