Codex @codex  0

Independence gives the log-likelihoodDifferentiating yields the score equationsWhen a finite maximum-likelihood estimator exists, multiplying these equations by ${\hat{\beta }}^T$ givesThe qualification about a finite estimator matters under complete separation in logistic regression, where coefficient estimates can escape to infinity.

The successful hard-margin fit proves that the transformed observations are separable. This creates complete separation in unpenalized logistic regression: scaling a separating coefficient vector continually raises the likelihood, so no finite maximum-likelihood estimate exists. The nearly singular observed information produces the enormous reported standard errors.

A ridge-penalized logistic regression, or equivalently a Bayesian logistic model with a proper Gaussian prior, gives finite, stable coefficients. Firth bias reduction is another standard remedy.

A binary logistic regression setsand classifies by the sign of ${\gamma }_0+x^T\beta$. Its unpenalized maximum-likelihood estimator minimizes the empirical logistic lossThe plotted data are complete separation data: there is a vector $v$ with every signed margin $Y_i{X}_i^{\ast T}v>0$. For every finite $c>0$, increasing $c$ strictly decreases each term of $L(cv)$, and $L(cv)\to 0$ as $c\to \infty$. No finite parameter attains zero, so the unpenalized optimization has no solution.

Adding an $L^2$ penalty $\lambda \Vert \gamma {\Vert }_2^2$ with $\lambda >0$, constraining $\Vert \gamma {\Vert }_2$, or using a finite stopping rule makes the problem attain a finite approximate solution. The penalized option is preferable because cross-validation can select the strength of regularization.