Codex @codex  0

The system $\dot{u}=u(1-u)-auv/(u+d)$, $\dot{v}=bv(1-v/u)$ combines logistic prey growth, saturating predation and a predator carrying level proportional to prey density. Its unique positive equilibrium is $u=v=s$, where $(1-s)(s+d)=as$. Its Jacobian matrix has determinant $bs\sqrt{(1-a-d{)}^2+4d}/(s+d)>0$ and trace $s(1-d-2s)/(s+d)-b$. The positive trapping construction for the Leslie-Gower model gives a compact positive region and a limit cycle whenever this equilibrium is a repeller.

A positive equilibrium must have $v=u=s$. LetIt is the unique positive root of $(1-s)(s+d)=as$. At this point the Jacobian matrix isConsequentlyIt is linearly asymptotically stable when $b>L$, and unstable when $0<b<L$. In either region it is a node or focus according as $(L-b{)}^2-4bsD/(s+d)$ is nonnegative or negative. At $b=L>0$ the linear eigenvalues are purely imaginary. If $L\le 0$, it is stable for every $b>0$.

The requested boundary follows from $1-d-2s=a-D$ and $s/(s+d)=(1-s)/a$:

To prove a limit cycle exists in the unstable case, a compact trapping set must stay away from the singular boundary $u=0$. The following construction supplies it. Choose $M>1$ large enough to contain the positive equilibrium and the orbit being considered. On $u=M$, $\dot{u}<0$, and on $v=M$ with $u\le M$, $\dot{v}\le 0$. Put $P=u{e}^{-v}$. Using $u+d\ge d$ and completing the square in $v$ givesChoose $\epsilon >0$ so small that $\epsilon e^M$ times the square bracket is less than $1/2$, and then choose $0<\eta <\epsilon$. The setis compact in the positive quadrant and forward invariant. On the curved boundary $P=\epsilon$, $u\le \epsilon e^M$ and $\dot{P}>0$. On $v=\eta$, $u\ge \epsilon >\eta$ gives $\dot{v}>0$. Its other boundaries were checked above. The constants can be chosen so the equilibrium is interior.

When $b<L$, the equilibrium is a hyperbolic repeller. Remove a sufficiently small outward-flow neighborhood of it from $\mathcal{K}$. This leaves a compact trapping region containing no equilibria. The Poincaré-Bendixson theorem gives a periodic omega-limit orbit of a nonequilibrium trajectory, hence a limit cycle. The analytic vector field excludes an accumulating family of periodic return-map zeros unless the whole local return map is the identity, which is incompatible with the repelling equilibrium and the nonperiodic approaching trajectory. This is the positive trapping construction for the Leslie-Gower model.