= Solution
A binary <logistic regression> sets
$$
\mathbb P(Y=1\mid X=x)
=\frac{1}{1+e^{-(\gamma_0+x^T\beta)}}
$$
and classifies by the sign of $\gamma_0+x^T\beta$. Its unpenalized <maximum-likelihood estimator> minimizes the empirical <logistic loss>
$$
L(\gamma)=\sum_{i=1}^n
\log\left(1+e^{-Y_iX_i^{*T}\gamma}\right).
$$
The plotted data are <complete separation> data: there is a vector $v$ with every signed margin $Y_iX_i^{*T}v>0$. For every finite $c>0$, increasing $c$ strictly decreases each term of $L(cv)$, and $L(cv)\to0$ as $c\to\infty$. No finite parameter attains zero, so the unpenalized optimization has no solution.
Adding an $L^2$ penalty $\lambda\|\gamma\|_2^2$ with $\lambda>0$, constraining $\|\gamma\|_2$, or using a finite stopping rule makes the problem attain a finite approximate solution. The penalized option is preferable because <cross-validation> can select the strength of <regularization>.
Back to article page
