Past exam of the mathematics course of the University of Cambridge / 2010 / iii / Paper 37 / 3

The Bernoulli distribution probability mass function isWriting ${\theta }_i=\log [p_i/(1-p_i)]$ gives $p_i=e^{{\theta }_i}/(1+e^{{\theta }_i})$ and $\log (1-p_i)=-\log (1+e^{{\theta }_i})$. Therefore the exponential family representation hasThe logit link is the canonical link.

Independence gives the log-likelihoodDifferentiating yields the score equationsWhen a finite maximum-likelihood estimator exists, multiplying these equations by ${\hat{\beta }}^T$ givesThe qualification about a finite estimator matters under complete separation in logistic regression, where coefficient estimates can escape to infinity.

For separate binary observations, the saturated model fits each outcome exactly and has maximized log-likelihood zero. With ${\hat{p}}_i=p_i(\hat{\beta })$, the residual deviance is thereforeUse the identity in part (b) to replace $y_i$ by ${\hat{p}}_i$ in its first term:Thus the deviance is twice the sum of the fitted binary information entropies. A deviance goodness-of-fit test against ${\chi }_{n-p}^2$ is unreliable for individual ungrouped binary observations: there is only one trial per saturated-model parameter, so the usual large-cell-count approximation fails. Deviance differences between fixed-dimensional nested logistic regression models can still support a likelihood-ratio test, subject to its usual regularity assumptions.

The three tree models are respectively $\mathrm{logit}{p}_i={\beta }_0$, $\mathrm{logit}{p}_i={\beta }_0+{\beta }_1{\log }_2{T}_i$, and $\mathrm{logit}{p}_i={\beta }_0+{\beta }_1{\log }_2{T}_i+{\beta }_2{S}_i$, with independent Bernoulli random variables conditional on the covariates. Testing the additional severity effect givesUnder $H_0:{\beta }_2=0$, its reference distribution is asymptotically ${\chi }_1^2$. The $p$-value is approximately $1.21\times 10^{-21}$, giving overwhelming evidence that severity improves the fitted model. Holding severity fixed, doubling $T$ increases ${\log }_{2}T$ by one and multiplies the odds byThis is an odds ratio, not a multiplication of the event probability itself.