Solution (source code)

= Solution

For positive $a,b$ and upward orientation, traverse the four edges counterclockwise. The bottom edge contributes $\int_0^a x^2dx=a^3/3$. The right edge contributes $\int_0^b2ay\,dy=ab^2$. On the top edge $x$ decreases, giving $\int_a^0(x^2-b^2)dx=-a^3/3+ab^2$. The left edge has $x=0$ and zero tangential component. Hence
$$
\boxed{\oint_{\partial S}\mathbf A\cdot d\mathbf r=\frac{a^3}{3}+ab^2-\frac{a^3}{3}+ab^2=2ab^2.}
$$
This agrees with the <surface integral> and explicitly verifies <Stokes theorem>, including the direction of the top-edge traversal.