Past exam of the mathematics course of the University of Oxford / 2023 / Prelims / Mathematics III / 3 / b

For a twice continuously differentiable function, an interior local extremum requires $f_x=f_y=0$. At such a critical point, put $D=f_{xx}{f}_{yy}-f_{xy}^2$. The second-derivative test gives a strict local minimum if $D>0$ and $f_{xx}>0$, a strict local maximum if $D>0$ and $f_{xx}<0$, and a saddle point if $D<0$. When $D=0$, the test is inconclusive. More generally the Hessian matrix is necessarily positive semidefinite at a local minimum and negative semidefinite at a local maximum, but these non-strict signs alone are not sufficient.

For this polynomial,Subtracting the two stationarity equations gives $y(2x-y)=0$. If $y=0$, then $x=0$; if $y=2x$, the remaining equation gives $x=0$ or $x=3/2$. Thus the only critical points are $(0,0)$ and $(3/2,3)$.

At $(3/2,3)$, $\det H=-18<0$, so it is a saddle point. At $(0,0)$ the determinant is zero, but along $y=-x$ the value is $-x^3$, which takes both signs arbitrarily near zero. ThereforeIn particular a vanishing Hessian determinant does not exclude a saddle point.