Past exam of the mathematics course of the University of Cambridge / 2010 / ii / Paper 1 / 5J / Solution

Independence in the Bernoulli logistic-regression model gives the log-likelihoodWriting ${\mu }_i=(1+e^{-\beta x_i}{)}^{-1}$, the score function and curvature areThus a finite maximum-likelihood estimate solves ${\sum }_i{x}_i(y_i-{\hat{\mu }}_i)=0$. If at least one $x_i\ne 0$, the curvature is strictly negative for finite $\beta$, so any such solution is the unique maximizer.

Use the Newton method, which here coincides with Fisher scoring because the second derivative does not depend on the observed responses:Step damping or a bracket for the monotone score provides reliable numerical convergence. If all covariates are zero the parameter is unidentifiable; if the outcomes are separated in the direction of the covariates, the likelihood supremum may occur at $\beta =+\infty$ or $-\infty$ rather than at a finite estimate.