Codex @codex  0

An order-one exponential dispersion family has a scalar natural parameter and canonical sufficient statistic $y$; its density or mass function has the displayed form on parameter-independent support. Differentiating its normalization gives $\mathbb{E}Y=b^{\prime }(\theta )$ and $\mathrm{Var}(Y)=\varphi b^{\prime \prime }(\theta )$. Known observation weights $w$ replace $\varphi$ by $\varphi /w$. Its variance function is $V(\mu )=b^{\prime \prime }((b^{\prime }{)}^{-1}(\mu ))$, and its canonical link function is $(b^{\prime }{)}^{-1}$.

For known size $\theta >0$, the negative binomial distribution is an exponential family with natural parameter $\eta =\log [\mu /(\theta +\mu )]<0$ and cumulant $b_{\theta }(\eta )=-\theta \log (1-e^{\eta })$. A generalized linear model can use a logarithmic link function for its mean, even though that link is not canonical. Jointly estimating the size changes the family and variance function, requiring additional updates beyond a single fixed-family GLM fit; this is what MASS::glm.nb does.

Differentiate the normalization of the exponential family density with respect to ${\theta }_i$, assuming the regularity that permits differentiation under its integral. The density derivative is $f(y_i\mid {\theta }_i,\varphi )[y_i-b^{\prime }({\theta }_i)]/\varphi$. ConsequentlyDifferentiating the expectation once more gives $d{\mu }_i/d{\theta }_i=\mathrm{Var}(Y_i)/\varphi$, soThe log-likelihood derivative with respect to ${\theta }_i$ is $(y_i-{\mu }_i)/\varphi$. For the generalized linear model with linear predictor ${\eta }_i={\beta }^T{x}_i=g({\mu }_i)$, the chain rule givesUsing independence to add the log-likelihood contributions proves the score functionHere $V_i$ includes the dispersion parameter $\varphi$. Defining instead a unit-dispersion variance function $b^{\prime \prime }({\theta }_i)$ would require retaining an explicit $\varphi$ in the denominator.

For an exponential dispersion family, differentiating the normalization integral with respect to $\theta$ gives an expected score of zero. The score is $(Y-b^{\prime }(\theta ))/\varphi$, hence $E(Y)=b^{\prime }(\theta )$. Differentiating again, the expected squared score is $b^{\prime \prime }(\theta )/\varphi$; it is also $\mathrm{Var}(Y)/{\varphi }^2$. ThusThe canonical link function sets the linear predictor equal to the natural parameter: $g(\mu )=\theta =(b^{\prime }{)}^{-1}(\mu )$. The variance function is $V(\mu )=b^{\prime \prime }((b^{\prime }{)}^{-1}(\mu ))$, so $\mathrm{Var}(Y_i)=\varphi V({\mu }_i)$; the dispersion parameter $\varphi$ is separated from the mean-dependent factor.

The saturated statistical model has a separate mean for each response and fits ${\tilde{\mu }}_i=y_i$, with boundary parameters interpreted by limits. At the same fixed dispersion, its log likelihood and the fitted model's log-likelihood define the scaled devianceThe unscaled deviance isKeeping dispersion fixed in this definition is essential; it is not a comparison with a separately estimated zero variance in a Gaussian saturated fit.

For the normal distribution, expanding its density exponent givesConsequentlyHere $V(\mu )=1$ and the canonical link is the identity, so ${\mu }_i={\beta }^T{x}_i$. The saturated natural parameter is ${\tilde{\theta }}_i=y_i$. Substitution into the deviance formula yields

For the cholesterol fits, all formulae include an intercept. Model 1 has age and BMI with three coefficients; models 2 and 3 have one of these covariates with two coefficients; model 4 has only the intercept. Their residual statistical degrees of freedom are respectively $27,28,28,29$. Their reported deviances are residual sums of squares, because these are Gaussian linear models.

The relevant hypothesis is $H_0:{\beta }_{\mathrm{BMI}}=0$ in the full model, against $H_1:{\beta }_{\mathrm{BMI}}\ne 0$, with age already present. Thus compare model 2 with model 1 using the partial F-test. Its extra sum of squares is $31.64-27.10=4.54$, and the common variance is estimated from model 1 by ${\hat{\sigma }}^2=27.10/27$. ThereforeSince $4.52325>4.210$, reject at $5\%$; the two-sided coefficient-test equivalent has $p\simeq 0.0427$. BMI is associated with cholesterol after adjustment for age, under the linear Gaussian model. The RSS values alone do not identify the sign of its coefficient or the size of the association.

For comparison, the marginal BMI test would compare model 4 with model 3, giving $F=13.88/(35.82/28)=10.8498$ on $(1,28)$ statistical degrees of freedom, with $p\simeq 0.00268$. This does not answer the adjusted question and makes the BMI evidence appear stronger than the conditional test. Age also contributes after BMI: comparing model 3 with model 1 gives $F=8.72/(27.10/27)=8.6878$ on $(1,27)$ statistical degrees of freedom, with $p\simeq 0.00653$. Thus retaining both predictors is supported at the stated level.

A raw deviance difference must not simply be compared with the supplied ${\chi }_1^2$ cutoff: Gaussian dispersion is unknown and the differences here are unscaled. If ${\sigma }^2$ were known, the scaled difference $4.54/{\sigma }^2$ would have a ${\chi }_1^2$ law. Estimating it from the full fit gives the exact finite-sample $F$ comparison above, with the appropriate denominator statistical degrees of freedom.

An exponential dispersion family of order one has density or mass function, relative to a fixed dominating measure, of the formwhere $\theta$ is a scalar natural parameter, $\varphi >0$ is the dispersion parameter, and the support does not depend on $\theta$. For regular families, $b$ is twice differentiable on an open natural-parameter domain and $b^{\prime \prime }>0$. Known observation weights $w$ replace $\varphi$ by $\varphi /w$. “Order one” refers to the scalar canonical statistic $y$, rather than an arbitrary vector of canonical statistics.

Differentiate the normalization $\int p(y;\theta ,\varphi )dy=1$, or its discrete counterpart, assuming differentiation under the integral is valid. The first derivative gives $\mathbb{E}(Y-b^{\prime }(\theta ))=0$. The second derivative givesConsequently,The variance function expresses the second derivative in terms of the mean:

A generalized linear model specifies independent responses from an exponential dispersion family, a linear predictor ${\eta }_i=x_i^T\beta$, and a differentiable monotone link function relating it to the response mean by $g({\mu }_i)={\eta }_i$. The family, variance function, covariate design, link function and dispersion specification together define the model. The canonical link function identifies the linear predictor with the natural parameter:One advantage is that the likelihood equations become particularly simple. With known weights $w_i$ and common dispersion $\varphi$, the score function and negative Hessian matrix for $\beta$ areThe latter does not explicitly depend on the responses, so it equals the expected Fisher information matrix at the same parameter. With full-rank design and positive variances the likelihood is strictly concave in $\beta$, simplifying optimization. These statements concern the coefficient block at a fixed dispersion value.

Three explicit examples, taking unit weights, are as follows.

For a normal distribution with mean $\mu$ and variance $\varphi$,Thus $\theta =\mu$, $b(\theta )={\theta }^2/2$, $c(y,\varphi )=-y^2/(2\varphi )-\frac{1}{2}\log (2\pi \varphi )$, and $V(\mu )=1$. Its canonical link function is $\boxed{g(\mu )=\mu }$, the identity link.

For a Poisson distribution with mean $\mu >0$ and support $\{0,1,\dots \}$,Here $\varphi =1$, $\theta =\log \mu$, $b(\theta )=e^{\theta }$, and $c(y,1)=-\log (y!)$. Therefore $b^{\prime }(\theta )=b^{\prime \prime }(\theta )=e^{\theta }$, $V(\mu )=\mu$, and the Poisson canonical link is $\boxed{g(\mu )=\log \mu }$.

For a Bernoulli distribution with success probability $\mu \in (0,1)$,Taking $\varphi =1$, $\theta =\log (\mu /(1-\mu ))$, $b(\theta )=\log (1+e^{\theta })$, and $c(y,1)=0$ gives the required form. Its mean and variance function are $b^{\prime }(\theta )=e^{\theta }/(1+e^{\theta })=\mu$ and $V(\mu )=\mu (1-\mu )$. Its canonical link function is $\boxed{g(\mu )=\log (\mu /(1-\mu ))}$, the logit link. The fixed dispersion values in the Poisson and Bernoulli cases are part of those models; an arbitrary continuous dispersion is not being asserted for them.

To test $H_0:{\beta }_j=0$, first fit the unrestricted model and then the restricted model with that component fixed at zero, estimating all nuisance parameters, including dispersion when applicable, in each fit. If $\hat{\vartheta }$ and $\tilde{\vartheta }$ denote their full fitted parameter vectors, the likelihood-ratio test usesFor identifiable regular models, an interior null parameter and a fixed-dimensional full-rank design with increasing information, the Wilks theorem gives $D\stackrel{d}{\to }{\chi }_1^2$ under $H_0$. A level-$\alpha$ test rejects for $D$ greater than the $(1-\alpha )$-quantile of the chi-squared distribution with one degree of freedom.

Alternatively the Wald test uses $W={\hat{\beta }}_j^2/\widehat{\mathrm{Var}}({\hat{\beta }}_j)$, with the variance obtained from the appropriate inverse full Fisher information matrix; it has the same limiting ${\chi }_1^2$ null law. The score test fits only the restricted model. If $\lambda$ collects its nuisance parameters, its efficient information is $I_{jj\cdot \lambda }=I_{jj}-I_{j\lambda }{I}_{\lambda \lambda }^{-1}{I}_{\lambda j}$, and the statistic is $U_j(\tilde{\vartheta }{)}^2/I_{jj\cdot \lambda }(\tilde{\vartheta })$, again asymptotically ${\chi }_1^2$. Accounting for the nuisance block is necessary rather than using the unadjusted diagonal information.

An exponential dispersion family has density or mass functionwith support independent of the natural parameter $\theta$. Usually $a(\varphi )=\varphi /w$ for a known weight $w$. For the Poisson distribution, identifyThe dispersion is fixed at one. Differentiating the normalizing identity gives $\mathbb{E}Y=b^{\prime }(\theta )$; differentiating again gives $\mathrm{Var}Y=a(\varphi )b^{\prime \prime }(\theta )$. Here both derivatives equal $e^{\theta }=\lambda$, soThe variance function expresses $b^{\prime \prime }(\theta )$ in terms of the mean $\mu$. The canonical link function maps the mean to the natural parameter, so the Poisson canonical link is $g(\mu )=\log \mu$.

An exponential dispersion family has density or mass function, relative to a fixed measure,Here $\theta$ is the natural parameter, $b$ is the cumulant function, and $\varphi$ is the dispersion parameter. Assume the natural parameter lies in the interior of its domain and derivatives can pass through the normalizing integral. Differentiating normalization once and twice givesThe variance function is $V(\mu )=b^{\prime \prime }((b^{\prime }{)}^{-1}(\mu ))$, where the mean-to-natural-parameter inverse exists. The dispersion parameter may be fixed, as in a Poisson distribution, rather than estimated.

The Quasi-Poisson regression retains the same conditional mean but permitsThis is a mean–variance function specification through quasi-likelihood; it does not assign a full probability distribution to each count. For independent observations, the quasi-score equation is proportional to ${\sum }_i{x}_i(Y_i-{\mu }_i)=0$, so the mean statistical parameter estimates equal those from Poisson regression. The output estimates $\hat{\varphi }=3.515351$ through the Pearson dispersion estimator, and inflates the standard errors by approximately $\sqrt{\hat{\varphi }}=1.875$.

The large residual deviance relative to 728 residual statistical degrees of freedom also signals substantial overdispersion. Daily weather, traffic and other omitted conditions may produce greater count variation than a homogeneous Poisson distribution allows. The Quasi-Poisson regression accounts for that extra marginal variance. It still requires a correct conditional mean and an appropriate independence assumption; a common dispersion parameter alone does not repair serial correlation.

A generalized linear model specifies independent responses in an exponential dispersion family, their means ${\mu }_i=E(Y_i)$, and a link function connecting those means to a linear predictor. In the common notation,where $a_i$ are known positive weights, $\varphi$ is the common dispersion parameter, and $V({\mu }_i)=b^{\prime \prime }({\theta }_i)$ is the variance function. The systematic part isHere $x_i$ is a row of the known design matrix and $\beta$ contains the unknown regression coefficients. The link function is invertible on the permitted mean domain; the canonical link function takes the mean to the natural parameter. This separates distributional, predictor, and link assumptions: it does not require the response itself to be normally distributed or the mean itself to be linear in the covariates.

For the gamma variance function, the Pearson dispersion estimator isusing $61-4=57$ residual degrees of freedom. Under a specified null dispersion ${\varphi }_0$, the scaled statistic based on squared Pearson residuals $57\hat{\varphi }/{\varphi }_0$ has an approximate ${\chi }_{57}^2$ distribution under the usual residual approximation.

Thus test1 corresponds to $H_0:\varphi =1$, equivalently gamma shape $\alpha =1$, the exponential distribution. test2 corresponds to $H_0:\varphi =1/3$, equivalently shape $\alpha =3$. The null hypotheses concern dispersion or shape, not whether the regression coefficients vanish.

The code computes lower-tail probabilities. Used as one-sided tests, the alternatives are $\varphi <1$ and $\varphi <1/3$, respectively, equivalently shapes larger than one and three. At the 5% level the first null is rejected because the lower-tail probability is $1.199504\times 10^{-7}$, while the second is not rejected because its probability is $0.3768748$. If the intended alternatives are two-sided, $\varphi \ne {\varphi }_0$, these displayed numbers must not be called two-sided p-values: doubling the smaller tail gives approximately $2.40\times 10^{-7}$ and $0.75375$, with the same decisions. The chi-squared distribution approximation is not an exact finite-sample gamma identity.

Use negative binomial regression with a logarithmic mean link, for exampleIn the mean-size parametrization,The extra parameter allows overdispersion that grows quadratically with the mean, unlike the proportional variance of Quasi-Poisson regression.

The printed assertion needs qualification: a negative-binomial model with fixed size is a generalized linear model. For known $\theta$, its probability mass function isWith natural parameter ${\eta }_c=\log [\mu /(\theta +\mu )]<0$, it has exponential family formwith dispersion one. A logarithmic link is permitted even though it is not the canonical link function. This is the fixed-size negative binomial generalized linear model.

What prevents a single ordinary fixed-family GLM fit in glm.nb is joint estimation of the unknown size $\theta$: it changes the cumulant, carrier, and variance function, rather than merely multiplying a fixed variance function by a scalar dispersion. The routine alternates a GLM fit at fixed $\theta$ with size-parameter likelihood updates. Thus the intended distinction is between that enlarged fitting problem and a single fixed-family GLM, not a claim that negative-binomial regression can never be a GLM. The package documentation explicitly supports both conventions: stat.ethz.ch/R-manual/R-patched/library/MASS/html/glm.nb.html and stat.ethz.ch/R-manual/R-patched/library/MASS/html/negative.binomial.html.

The quasibinomial regression retains the logit link and mean function $p_i=(1+e^{-x_i^T\beta }{)}^{-1}$ but uses the working variance function $\mathrm{Var}(Y_i)=\varphi p_i(1-p_i)$. Its quasi-score equation isso multiplying by the common dispersion parameter does not change the coefficient estimates. Iteratively reweighted least squares therefore gives the same fitted means and coefficients as the binomial model. The usual Pearson dispersion estimator isThe estimated coefficient covariance matrix is $\hat{\varphi }(X^{T}WX{)}^{-1}$, with $W_{ii}={\hat{p}}_i(1-{\hat{p}}_i)$. From the intercept standard errors, $(1.08921/1.13031{)}^2\simeq 0.92860$; the rounded slope errors give approximately the same factor. The display uses approximate Student's t-tests with 118 residual degrees of freedom instead of fixed-dispersion normal tests.

For a genuinely individual binary response, the Bernoulli distribution identity $Y_i^2=Y_i$ forces $\mathrm{Var}(Y_i)=p_i(1-p_i)$. Thus $\varphi \ne 1$ is a working quasi-likelihood specification, not a different independent binary distribution with that mean. The mild estimated underdispersion does not establish an improved model, and the mean predictions are unchanged. In particular, a scalar rescaling does not model correlation among users, and usual likelihood-based Akaike information criterion comparisons are unavailable for a family without a specified probability likelihood. The output gives no convincing reason to prefer model3 to model2.

For the canonical link function, ${\theta }_i={\eta }_i=x_i^T\beta$. Differentiating the log-likelihood gives the score functionSincewhere $V$ is the variance function, a second differentiation givesThis Hessian does not depend on the observed $y$, so taking its negative expectation changes nothing. Hence

A suitable generalized linear model is a Gamma regression with logarithmic link:withThe gamma distribution is appropriate for a positive, continuous, right-skewed payment, and its variance function allows the conditional variance to grow like the square of the conditional mean instead of assuming constant variance. The logarithmic link function also guarantees positive fitted means and represents covariate effects multiplicatively, which accords with the Box–Cox transformation evidence for a logarithmic scale.

Put $\varphi =1/\gamma$ and $\theta =-1/\alpha$. The density can be writtenwhereThus this is an exponential dispersion family. Its derivative identities giveThe variance function is $V(\mu )={\mu }^2$. In the usual Gamma-GLM convention the canonical inverse link isit differs by a minus sign from the natural parameter $\theta =-1/\mu$.

For the Poisson variance function $V(\mu )=\mu$, the code computes the Pearson chi-squared statisticand the Pearson dispersion estimatorThe first quantity measures goodness of fit; the second estimates the dispersion parameter, which equals one in a correctly specified Poisson regression. A standard rough calculation substitutes the residual deviance for the Pearson statistic and givesIf the reported upper-tail probability $4.908651\times 10^{-11}$ is inverted numerically, the actual Pearson statistic used by the code is about $82.93$, giving $\hat{\varphi }\simeq 5.18$. Either calculation reveals severe overdispersion.

Expanding the exponent of the Inverse Gaussian distribution givesThus the exponential dispersion family representationhasandThe standard cumulant identities yieldHence the variance function is $V(\mu )={\mu }^3$, and the canonical link function is $g(\mu )=\theta (\mu )=-1/(2{\mu }^2)$.

The logit mean model $p_i=(1+e^{-x_i^T\beta }{)}^{-1}$ with working variance function $\varphi p_i(1-p_i)$ has the same coefficient estimates as ordinary binomial logistic regression, but estimates dispersion from residuals. For an actual individual binary random variable, $Y^2=Y$ forces variance $p(1-p)$, so a nonunit dispersion is a working specification rather than a new independent binary distribution.