Solution (source code)

= Solution

At a regular point of the constraint curve, $\nabla g\ne0$. A constrained stationary point has $Df$ zero on every tangent vector annihilated by $Dg$, so its gradient is normal to the curve. The <Lagrange multiplier> equations are therefore $\nabla f=\lambda\nabla g$ together with $g=0$. Singular constraint points require separate examination.

Assume $a,b>0$. For the ellipse, the equations are
$$
y=\frac{2\lambda x}{a^2},\qquad x=\frac{2\lambda y}{b^2},\qquad\frac{x^2}{a^2}+\frac{y^2}{b^2}=1.
$$
Neither coordinate can vanish at a stationary point, because the first two equations would then force both to vanish. Multiplication gives $4\lambda^2=a^2b^2$. Therefore $y=\pm(b/a)x$, and the constraint gives $x^2=a^2/2$, $y^2=b^2/2$. The stationary values are
$$
\boxed{xy=ab/2\text{ at }(a/\sqrt2,b/\sqrt2),(-a/\sqrt2,-b/\sqrt2),}
$$
$$
\boxed{xy=-ab/2\text{ at }(a/\sqrt2,-b/\sqrt2),(-a/\sqrt2,b/\sqrt2).}
$$
The compact ellipse has global extrema, all its points are regular, and these are all stationary points. Hence the first value is the global maximum and the second the global minimum.