= Solution
For <logistic regression>, set $\eta_i=x_i^{\mathsf T}\beta$ and $p_i=(1+e^{-\eta_i})^{-1}$. Its Bernoulli <log-likelihood>, <score function> and <Fisher information> are
$$
\ell(\beta)=\sum_i\{y_i\eta_i-\log(1+e^{\eta_i})\},\qquad
U(\beta)=X^{\mathsf T}(y-p),\qquad
I(\beta)=X^{\mathsf T}WX,
$$
where $W=\operatorname{diag}\{p_i(1-p_i)\}$. Solve $U(\beta)=0$ by <Fisher scoring> or <iteratively reweighted least squares>. One scoring step is
$$
\boxed{\beta^{\rm new}=\beta+(X^{\mathsf T}WX)^{-1}X^{\mathsf T}(y-p)}.
$$
Equivalently regress the working response $z_i=\eta_i+(y_i-p_i)/[p_i(1-p_i)]$ on $X$ using weights $p_i(1-p_i)$. Iterate until the <likelihood> and coefficients stabilize, with rank and convergence checks. <Separation in logistic regression> can prevent a finite maximum: if a predictor direction strictly separates the zeros and ones, moving indefinitely along it improves the <likelihood> rather than producing a finite root.
Under the unrestricted model, each individual <probability> is estimated independently as $\widehat p_i=y_i$. Every observed binary outcome then has <probability> one, so, using boundary limits,
$$
\boxed{\max_{0\le p_i\le1}\ell(p_1,\ldots,p_n)=0}.
$$
The resulting <residual deviance> is $D=-2\ell(\widehat\beta)$. It is a legitimate <likelihood> contrast and can be used to compare fitted models. However, <individual Bernoulli deviance need not have a chi-squared calibration>: the saturated model has one boundary-valued parameter for each single observation, and its dimension grows with $n$. The fixed-dimensional regularity behind <Wilks theorem> does not justify a $\chi^2_{n-p}$ reference here.
For a concrete demonstration, let the true model contain only an intercept and have success <probability> $1/2$. Its fitted <probability> is $\bar y$, and
$$
\frac Dn=-2\{\bar y\log\bar y+(1-\bar y)\log(1-\bar y)\}
\longrightarrow2\log2,
$$
whereas a $\chi^2_{n-1}$ variable divided by $n$ converges to one. Thus treating the raw Bernoulli <residual deviance> as an ordinary absolute goodness-of-fit statistic can reject a correct model systematically. In contrast, <deviance> differences between fixed-dimensional nested logistic models have their usual asymptotic chi-squared reference under the relevant regularity assumptions. For absolute fit use meaningful replicated covariate groups and <grouped-binomial logistic regression>, or assess residual patterns and prediction on held-out data; a model-specific <parametric bootstrap> can calibrate a chosen statistic.
Back to article page
