= Solution
{c}
Use the <change of variables> $u=x^2-y^2$, $v=xy$. Its <Jacobian determinant> in the first quadrant is $2(x^2+y^2)>0$, and the inverse is uniquely determined by
$$
x^2=\frac{\sqrt{u^2+4v^2}+u}{2},\qquad
y^2=\frac{\sqrt{u^2+4v^2}-u}{2}.
$$
Both are positive for $v>0$, so the transformation is one-to-one on the region. It becomes the <rectangle> $-2\leq u\leq4$, $1\leq v\leq3$. The <integrand> is $uv$, and $x^2+y^2=\sqrt{u^2+4v^2}$. Thus
$$
\begin{aligned}
\iint_A(x^3y-xy^3)\,dx\,dy
&=\int_1^3\int_{-2}^4\frac{uv}{2\sqrt{u^2+4v^2}}\,du\,dv\\
&=\int_1^3v\left(\sqrt{v^2+4}-\sqrt{v^2+1}\right)\,dv\\
&=\boxed{\frac{13\sqrt{13}-10\sqrt{10}-5\sqrt5+2\sqrt2}{3}.}
\end{aligned}
$$
Back to article page