Past exam of the options II course of the University of Oxford / 2006 / F / 2 / i / Solution

The Fréchet derivative at $a$ is the linear map $D{f}_a:{\mathbb{R}}^n\to {\mathbb{R}}^k$ satisfyingIt is unique. When both derivatives exist, the chain rule is $D(g\circ f{)}_a=D{g}_{f(a)}\circ D{f}_a$.

For the square map,The remainder has norm at most $\Vert \Delta {\Vert }^2$ in a submultiplicative matrix norm, hence is $o(\Vert \Delta \Vert )$. ThereforeMatrix multiplication is not commutative, so the two terms cannot generally be combined as $2A\Delta$.