Solution (source code)

= Solution

There is no stable positive coexistence state: the only one is a saddle for every $\rho,\alpha>0$ when $\beta>1$. Moreover, the <Bendixson-Dulac criterion> with Dulac function $B(x,y)=1/(xy)$ applies in the positive quadrant, since
$$
\nabla\cdot(BF)
=\frac{\partial}{\partial x}\left(\frac{\rho(1-x-\alpha y)}y\right)
+\frac{\partial}{\partial y}\left(\frac{1-\beta x}{x}\right)
=-\frac{\rho}{y}<0.
$$
Thus no periodic orbit can support long-term coexistence. Except on the saddle's stable manifold, one species is excluded: species 2 becomes extinct for initial data in the basin of $(1,0)$, ecologically when species 1 is sufficiently abundant and species 2 sufficiently scarce. On the other side, species 1 becomes extinct and the unregulated second species grows.