Past exam of the mathematics course of the University of Cambridge / 2017 / iii / Paper 206 / 1

Write the exponential dispersion family in the conventionAssume that the support does not depend on the parameters, that $\theta$ is in the interior of the natural parameter space, and that differentiation can pass under the normalizing integral or sum. Differentiating $\int f=1$ at fixed dispersion parameter givesso the expectation is $b^{\prime }(\theta )$. A second differentiation givesConsequentlyHere $b$ is the cumulant function of an exponential family; the same identities follow by differentiating $\log \mathbb{E}{e}^{tZ}=[b(\theta +\varphi t)-b(\theta )]/\varphi$. For a known observation weight $w$, replace $\varphi$ by $\varphi /w$. These regularity assumptions for differentiation under the integral sign matter: differentiation of a parameter-dependent support can introduce additional boundary terms.

For fixed known $r>0$, take the natural parameter of an exponential familyThe negative binomial distribution has log-likelihoodThus this is a negative binomial exponential family, with dispersion parameter $\varphi =1$ in the convention of part (a). The carrier contains the Gamma function and depends on the fixed size $r$, which need not be an integer. Differentiating the cumulant function of an exponential family givesThereforeThe size $r$ controls overdispersion relative to a Poisson distribution; it is different from the unit dispersion parameter used by this fixed-size generalized linear model. Another common convention applies to the scaled response $Z=Y/r$: then $b(\theta )=-\log (1-e^{\theta })$ and $\varphi =1/r$. Keeping track of the response scale prevents confusing these two representations. At $\lambda =0$ the law is concentrated at zero and is obtained as a boundary limit, rather than by a finite natural parameter of an exponential family.

The negative binomial regression assumes independent random variables conditional on their recorded days, withThe logarithmic link function applies to the conditional expectation, not to the observed count. The displayed maximum-likelihood estimates areand the dispersion parameter for the fitted family is one. Thus the fitted variance is ${\hat{\lambda }}_i+{\hat{\lambda }}_i^2/\hat{r}$, practically the Poisson distribution variance at these means.

Increasing the day by one multiplies the conditional expectation by $e^{{\hat{\beta }}_1}\simeq 0.996836$, a decrease of about $0.3164\%$. Comparing days 365 and 1 gives $e^{364{\hat{\beta }}_1}\simeq 0.3155$. The negative slope has Wald statistic approximately $-35.79$, so under the stated model assumptions there is very strong evidence of a decreasing conditional mean:This is an association with calendar day, not a causal effect established by the regression. The printed negative-binomial residual deviance is $7544.3$ on $1298$ residual degrees of freedom, which is a reason to investigate the adequacy of the assumed count model and dependence structure. In particular, the small reported slope standard error cannot make a misspecified model reliable.

Introduce a latent Bernoulli distribution indicator $A_i$, with $A_i=1$ denoting a structural zero and $\mathrm{Pr}(A_i=1)=\pi$. Conditional on $A_i=0$, retain the negative binomial regression with ${\lambda }_i=e^{x_i^T\beta }$ and $x_i=(1,d_i{)}^T$. This gives a zero-inflated negative binomial model:A participant who fishes may still catch zero, so an observed zero does not reveal the latent class. This distinguishes zero inflation from a hurdle model, which truncates the count component at zero. The marginal expectation becomes $(1-\pi ){\lambda }_i$, and the law of total variance givesA constant $\pi$ is a parsimonious starting model. If there is evidence that nonparticipation changes with day or other recorded predictors, use logistic regression for ${\pi }_i$ instead. Conditional independence of visitors remains an assumption; extra zeros alone do not establish its validity.

Let $I_0=\{i:y_i=0\}$ and $I_{+}=\{i:y_i>0\}$. Under the proposed zero-inflated negative binomial model, independence gives the likelihood functionThe positive-count factor uses the ordinary, untruncated negative binomial distribution. With $f_i=f_{\mathrm{NB}}(y_i;r,{\lambda }_i)$, the log-likelihood isFor latent indicators $a_i$, the complete-data log-likelihood iswhere $a_i=1$ is allowed only when $y_i=0$. This separation is what makes the expectation-maximization algorithm convenient.

For EM for zero-inflated negative binomial regression, at iteration $k$ apply Bayes' theorem to each zero in the zero-inflated negative binomial model. The E step computes the conditional expectation of its latent indicator:Treating these values as fixed, the M step maximizes the expected complete-data log-likelihood. Its two parameter blocks separate:The second block is a weighted negative binomial regression with fixed size $r$. Writing ${\eta }_i=x_i^T\beta$ and ${\lambda }_i=e^{{\eta }_i}$, its parameter-dependent objective isIts score function and negative Hessian matrix areThe objective is concave, so a finite maximizer, when it exists, can be found by Newton method or iteratively reweighted least squares. The latter uses working response ${\eta }_i+(y_i-{\lambda }_i)/{\lambda }_i$ and weights $w_{i}r{\lambda }_i/(r+{\lambda }_i)$. This is not an unweighted regression on only the positive counts: zeros retain fractional weight in the count component.

Start with $0<\pi <1$ and finite coefficients, alternate the two steps, and monitor the observed log-likelihood. Exact maximization makes the expectation-maximization algorithm nondecreasing in that likelihood, although the mixture may have multiple stationary points. All-zero data can lead to boundary fits and poor identification; multiple initializations and explicit checks for boundary parameters are useful.