Past exam of the mathematics course of the University of Cambridge / 2008 / ii / Paper 1 / 5J / Solution

The inverse logit link is ${\mu }_i(\beta )=e^{{\beta }^T{x}_i}/(1+e^{{\beta }^T{x}_i})$. independence therefore gives the Likelihood function and log-likelihoodDifferentiation yields the score ${\nabla }_{\beta }\ell ={\sum }_i{x}_i(y_i-{\mu }_i)$. The matrix whose rows are $x_i^T$ has dimension $n\times p$, not the PDF's printed $p\times n$. With this necessary dimension correction, any finite maximum Likelihood function estimate satisfiesThe Hessian matrix is $-X^{T}WX$, with $W_{ii}={\mu }_i(1-{\mu }_i)$, so the log-likelihood is concave. These score equations apply to a finite interior estimate; complete or quasi-complete separation can instead make its supremum occur only as coefficients diverge.

The deviance is twice the difference between maximized saturated and fitted log-likelihoods. A saturated Bernoulli distribution has fitted means $y_i\in \{0,1\}$ and log-likelihood zero, interpreted by limits. Hence the binomial deviance here isZero-multiplier logarithms are interpreted as zero.