 Solution

An interior stationary point has $f_x=f_y=0$. Assuming twice continuous differentiability nearby, the two-dimensional Taylor series about it iswhere $A=f_{xx}$, $B=f_{xy}$ and $C=f_{yy}$. Let $D=AC-B^2$. The second-derivative test gives a strict local minimum if $D>0,A>0$, a strict local maximum if $D>0,A<0$, and a saddle point if $D<0$.

For example, if $A\ne 0$, the quadratic form is $A(h_x+B{h}_y/A{)}^2+(D/A)h_y^2$, which proves these sign tests. If $D=0$, the quadratic part is inconclusive: $x^4+y^4$ has a minimum while $x^4-y^4$ has a saddle at the origin, although both have zero Hessian matrix there. Higher-order terms or direct inequalities must then be examined.