= Hessian matrix
{c}
{title2=$\operatorname{Hess} f$}
{wiki}
= Hessian
{c}
{synonym}
The <Hessian matrix> describes how the slope of a <function> changes in different directions. For a real-valued <function> $f$ with continuous second <partial derivatives> on an <open subset> of $\mathbb R^n$, it is
$$
H_f(x)=\left(\frac{\partial^2 f}{\partial x_i\partial x_j}(x)\right)_{i,j=1}^n.
$$
The <symmetry of second derivatives> makes $H_f$ a <symmetric matrix> under these hypotheses. It is also the <Jacobian matrix> of the <gradient>: near $x$, a small displacement $h$ changes $\nabla f$ by $H_f(x)h+o(\|h\|)$. Thus the <gradient> measures the current slope, while the <Hessian> measures its local variation. https://ocw.mit.edu/courses/18-s096-matrix-calculus-for-machine-learning-and-beyond-january-iap-2023/mit18_s096iap23_lec12.pdf[MIT 18.S096 matrix calculus notes, §12.1: Hessian matrices of scalar-valued functions]{ref}
For example, take the <quadratic form>
$$
q(x,y)=x^2+2xy+3y^2.
$$
Its <gradient> and <Hessian matrix> are
$$
\nabla q=(2x+2y,2x+6y)^T,
\qquad
H_q=\begin{pmatrix}2&2\\2&6\end{pmatrix}.
$$
The only <critical point> is the origin. Completing the square gives $q=(x+y)^2+2y^2$, so the origin is a strict <global minimum>. The <eigenvalues> $4-2\sqrt2$ and $4+2\sqrt2$ are both positive, recovering the same conclusion from the <positive definiteness> of the <Hessian>. Its <eigenvectors> identify the rotated directions in which the <quadratic form> has its smallest and largest bending.
\Image[/hessian-quadratic-critical-points.png]
{title=Positive, negative and mixed Hessian eigenvalues produce different critical-point geometry}
{description=Contour plots of $q=x^2+2xy+3y^2$, $-q$, and $s=x^2-y^2$. All three have a <critical point> at the origin; the first two have elliptical contours, while the last has positive and negative sectors meeting along its zero contours.}
For a fixed <vector> $v$, restriction to a <straight line> gives the directional interpretation
$$
\left.\frac{d^2}{dt^2}f(x+tv)\right|_{t=0}=v^TH_f(x)v.
$$
This is the <quadratic form> associated with the <Hessian matrix>. For a unit <eigenvector> $v$, it equals the corresponding <eigenvalue>. Opposite signs therefore mean that some directions bend upwards and others downwards. The <second-derivative test> turns this information into a classification when the <gradient> vanishes.
The same object is useful for computation. <Newton's method in optimization> chooses a displacement $h$ by solving $H_f(x)h=-\nabla f(x)$ when the <Hessian matrix> is an <invertible matrix>; this makes the <gradient> of the local quadratic model vanish. For the displayed $q$, the equation gives $h=-(x,y)^T$, so one exact step reaches the minimum from any starting point. For a general <function>, a <Hessian> with opposite-sign <eigenvalues> or a large <condition number> need not give a useful minimizing step, and the <local minimum> need not be a <global minimum>.
The name honors Otto Hesse. https://web.mit.edu/course/other/wwmath/vectorc/minmax/hessian.html[MIT World Web Math: the Hessian and its namesake]{ref} An older geometric use is the <Hessian criterion for flexes of a plane cubic>, connecting a <determinant> of second <partial derivatives> with inflection points of an algebraic <curve>. Hesse's 1844 paper \i[Über die Wendepuncte der Curven dritter Ordnung] is an early source on that geometric problem. https://www.degruyterbrill.com/document/doi/10.1515/crll.1844.28.97/html[Hesse (1844), Journal für die reine und angewandte Mathematik 28, pp. 97–107: publisher's bibliographic record]{ref}
Back to article page
