 Solution

At a stationary point, the linear terms vanish, so $\Delta f\simeq \frac{1}{2}(A{h}^2+2Bhk+C{k}^2)$, with $A=f_{xx}$, $B=f_{xy}$, $C=f_{yy}$. Let $D=AC-B^2$ be the Hessian determinant. For $A\ne 0$, completing the square givesThe second-derivative test therefore yieldsWhen $D<0$ the quadratic form is indefinite, including the case $A=0$, $B\ne 0$. When $D=0$, the quadratic approximation is degenerate and the test is inconclusive: higher-order terms may produce a minimum, maximum or saddle. For nondegenerate cases the quadratic term dominates the Taylor remainder sufficiently close to the point.