Solution (source code)

= Solution

Adding the two equations gives $\dot x+\dot y=b-x$, so a fixed point has
$$
(x_0,y_0)=\left(b,\frac{b}{a+b^2}\right).
$$
At this point the Jacobian has <determinant> $a+b^2>0$ and trace
$$
T=-1-a-b^2+\frac{2b^2}{a+b^2}.
$$
The fixed point is asymptotically stable exactly when $T<0$, or
$$
(a+b^2)(1+a+b^2)>2b^2.
$$
The equality curve exists only for $a\leq1/8$ and is
$$
b^2=\frac{1-2a\pm\sqrt{1-8a}}2.
$$
For $0<a<1/8$ the fixed point is unstable between these two curves and stable outside them; for $a>1/8$ it is stable everywhere.