Past exam of the mathematics course of the University of Oxford / 2017 / Prelims / Mathematics III / 3 / a

Write $h=x-x_0$, $k=y-y_0$, and let $a=f_{xx}$, $b=f_{xy}$ and $c=f_{yy}$ be evaluated at the critical point. Assuming $f$ is twice continuously differentiable nearby, the Taylor expansion isThe linear terms vanish because the gradient is zero. For $a>0$, completing the square givesConsequently the requested constant is the Hessian determinantThe displayed identity in the question is an identity for the quadratic part; for a general $f$, its right-hand side needs the remainder $o(h^2+k^2)$.

If $D>0$, the quadratic form is positive for every nonzero $(h,k)$. On the unit circle it has a positive minimum, so it dominates the remainder sufficiently near the critical point. Hence the point is a strict local minimum.

If $D<0$, along $k=0$ the quadratic term is $a{h}^2/2>0$, whereas along $h=-bk/a$ it is $D{k}^2/(2a)<0$. Along each line the remainder is of smaller order. Thus values both above and below $f(x_0,y_0)$ occur arbitrarily nearby, giving a saddle point.