Separation (statistics)

Separation in a binary regression model occurs when an affine score assigns all observations from one class to one side of a threshold and all observations from the other class to the other side.

In a binary logistic regression, write the signed directional margin of observation $i$ as $(2y_i-1)x_i^{T}d$. Quasi-complete separation occurs when a nonzero direction $d$ makes all these margins nonnegative, with some zero and some strictly positive. Along coefficients $\beta +td$, $t\to \infty$, each observation's log-likelihood contribution is nondecreasing, and those with positive margins strictly increase toward their limiting value. Hence the unpenalized fit cannot have a finite maximum in that direction. A group containing only outcome zeros is a simple example: sending its indicator coefficient to $-\infty$ improves that group's likelihood while leaving all other observations unchanged. Zero fitted risk is then a boundary limit, and ordinary finite-coefficient Wald confidence intervals are unavailable.

Complete separation means that every signed training margin is strictly positive for some affine score. In unpenalized logistic regression it causes coefficient norms to diverge and prevents a finite maximum-likelihood estimate.