 Solution

A Lyapunov function near an equilibrium is a continuously differentiable positive-definite function $V$, vanishing there, whose derivative along trajectories is nonpositive. The strict Second Lyapunov theorem says a negative-definite derivative implies local asymptotic stability.

For sharper bounds than the circular test $x^2+y^2$, put $z=(x,y{)}^T$, $r^2=z^{T}z$ andA direct multiplication gives $A^{T}P+PA=-2I$. Its eigenvalues are ${\lambda }_{\pm }=(13\pm \sqrt{13})/12>0$. Since the nonlinear vector field is $Az+r^{2}z$,For $0<V<1$, the derivative is strictly negative. The Lyapunov theorem proves asymptotic stability. In fact $V(0)<1$ gives $V(t)\le V(0)$ andso the solution stays bounded, exists for all positive time and converges exponentially to zero.

The ellipse $V=1$ is invariant. In polar coordinates, $\dot{\theta }=-(2{\cos }^2\theta +{\sin }^2\theta )<0$, so it is a nonconstant periodic orbit, not part of the basin. For $V(0)>1$, use $r^2\ge V/{\lambda }_{+}$ to obtain $\dot{V}\ge 2V(V-1)/{\lambda }_{+}$. This implies finite-time blow-up by comparison with the scalar equation. Thus the exact basin is $\{V<1\}$ and every point outside its closure escapes to unbounded norm within its maximal existence interval.

The largest open disc centred at zero lying in this exact basin has radiusThe smallest radius such that every point strictly outside its circle escapes is the largest radius on the invariant ellipse:These are optimal: a larger attraction disc contains a periodic-orbit point, and a smaller escape threshold leaves a periodic-orbit point in its exterior. This is an exact quadratic basin boundary.

The simpler radial derivative is $\frac{1}{2}(d/dt)r^2=r^2(r^2-1)-xy$. It only certifies attraction for $r<1/\sqrt{2}$ and strictly increasing norm for $r>\sqrt{3/2}$. Those are optimal thresholds for a sign test on that radial derivative, not for the actual attraction and escape regions. If “increases without bound” is read as requiring monotonic increase from the initial instant, the smallest such uniform radius is $\sqrt{3/2}$: at direction $x=y$, any smaller threshold permits an initially decreasing norm. The sharper $r_2$ above concerns eventual unbounded escape, which need not be monotone at every instant.