Past exam of the options II course of the University of Oxford / 2013 / F2

The total derivative of $F$ at $x$ is the linear map $D{F}_x:{\mathbb{R}}^n\to {\mathbb{R}}^m$ such thatThe Jacobian matrix is the $m\times n$ matrix $J_F(x)=(\partial F_i/\partial x_j)(x)$. When the total derivative exists, this is its matrix in standard bases: $D{F}_x(h)=J_F(x)h$. Existence of partial derivatives alone need not imply existence of the total derivative.

The map is continuously differentiable if it is differentiable at every point of $U$ and the derivative map $x\mapsto D{F}_x$ is continuous. In finite dimensions this is equivalent to all first partial derivatives of all its components existing and being continuous on $U$. The class is denoted $C^1(U,{\mathbb{R}}^m)$.

If $F:U\subseteq {\mathbb{R}}^n\to {\mathbb{R}}^m$ is differentiable at $x$, and $G$ is defined on a neighborhood of $F(x)$ and differentiable there, the chain rule statesIn standard coordinates the same statement isThe evaluation point in the first factor is $F(x)$, and the matrix order follows the order in which the linear maps act on increments.

For an increment $h$, direct expansion and the chain rule giveFor example, $\Vert x+h{\Vert }^2-\Vert x{\Vert }^2=2x^{T}h+\Vert h{\Vert }^2$, so the quadratic remainder divided by $\Vert h\Vert$ tends to zero. The corresponding gradients are $\nabla g(x)=2x$ and $\nabla k(x)=2A^{T}Ax$; the derivative of $F$ is the constant matrix $A$.

The Lagrange multiplier theorem for one regular constraint says: if $f,k$ are $C^1$ near $x$, $f(x)=0$, $\nabla f(x)\ne 0$, and $x$ is a local extremum of $k$ restricted to $f^{-1}(0)$, thenFor several constraints with linearly independent gradients, the objective gradient lies in their linear span. The regularity assumption rules out singular points where the usual multiplier conclusion can fail.

On the constraint set, $\Vert x\Vert =1$, so $\nabla f(x)=2x\ne 0$. Part (a)(iv) and the Lagrange multiplier theorem give $2A^{T}Ax=2\lambda x$, henceSince $x$ is nonzero, it is an eigenvector. Moreover $\lambda =x^T{A}^{T}Ax=\Vert Ax{\Vert }^2\ge 0$, so the associated eigenvalue is real and nonnegative.

For $n\ge 1$, the unit sphere is nonempty and compact by the Heine-Borel theorem. The function $k(x)=\Vert Ax{\Vert }^2$ is continuous and therefore attains a minimum and a maximum on it by the extreme value theorem. Either point supplies a nonzero eigenvector in part (ii), with a real multiplier. Thus $A^{T}A$ always has a real eigenvalue, indeed a nonnegative one. This gives existence without assuming the spectral theorem first.