Fréchet derivative

The Frechet derivative of $f$ at $a$ is the unique linear map $D{f}_a$ such that

For a positive integer $m$, the negative matrix power is differentiable on the open general linear group. The Neumann series gives $(A+H{)}^{-1}=A^{-1}-A^{-1}H{A}^{-1}+O(\Vert H{\Vert }^2)$ locally around any invertible matrix. Applying the product rule to the $m$ factors of this inverse gives the displayed Fréchet derivative. Each summand chooses one inverse factor to differentiate while keeping the others in order.

The factor order matters. Only when $A$ and $H$ commute does the sum simplify to $-m{A}^{-(m+1)}H$. This is a useful example of why scalar differentiation rules for powers cannot be transferred to matrices without accounting for noncommutativity.

For $I(u)={\int }_a^{b}f(x,u,u^{\prime })dx$ on $W^{1,\infty }$, continuity of $f,f_u,f_v$ on $[a,b]\times {\mathbb{R}}^2$ gives a bounded Fréchet derivative in the displayed form. Fixing $u$ and restricting $\Vert w{\Vert }_{W^{1,\infty }}$ to a bounded neighborhood confines all intermediate pairs $(u+tw,u^{\prime }+t{w}^{\prime })$ to a compact range up to null sets. Uniform continuity of the partial derivatives bounds the integrated first-order remainder by $(b-a)\omega (\Vert w\Vert )(\Vert w{\Vert }_{\infty }+\Vert w^{\prime }{\Vert }_{\infty })$, where $\omega (s)\to 0$. This is little-oh of the Sobolev norm. The result needs no continuous derivative representative for $u$; essential boundedness suffices.

Suppose $f(a+h)=f(a)+Lh+r(h)$ and $|r(h)|/\Vert h{\Vert }_1\to 0$. If $c\Vert h{\Vert }_1\le \Vert h{\Vert }_2\le C\Vert h{\Vert }_1$, then $|r(h)|/\Vert h{\Vert }_2\le c^{-1}|r(h)|/\Vert h{\Vert }_1\to 0$. The boundedness of $L$ is also preserved, and the same argument in reverse proves equivalence of differentiability. The derivative is the same linear map, not a norm-dependent replacement.

A map on a subset of a normed vector space is Hadamard differentiable at $s$ if its directional difference quotients converge to a continuous linear map uniformly along all convergent sequences of directions: $t_j\to 0$, $a_j\to a$, and $s+t_j{a}_j$ in the domain imply $(\Phi (s+t_j{a}_j)-\Phi (s))/t_j\to {\dot{\Phi }}_s(a)$. Tangential differentiability restricts the limit directions to a specified subspace. This is the regularity used by the functional delta method.

For a Hadamard differentiable functional with continuous linear derivative $L$, define $R_t(h)=(\Phi (s+th)-\Phi (s))/t-L(h)$. For every compact $K$ and positive $ϵ$, some positive $\delta ,a$ make $|R_t(h)|<ϵ$ whenever $\mathrm{dist}(h,K)<\delta$ and $0<|t|<a$. If this failed, choose $t_j\to 0$, directions within $1/j$ of $K$ and remainders at least $ϵ$. Compactness gives a convergent subsequence of nearby points of $K$, hence convergent directions. The derivative definition and continuity of $L$ give remainder tending to zero, a contradiction. With a tight weak limit, the Portmanteau theorem makes random directions lie near a suitable compact set with arbitrarily large probability. Thus this lemma proves the functional delta method without assuming separability of the entire normed space or uniform differentiability on its whole unit ball.

On bounded continuously differentiable functions with uniformly bounded total variation of a function, the map $T$ is Hadamard differentiable in the supremum norm with derivative ${\dot{T}}_{F,G}(a,b)={\int }_0^{u}b{F}^{\prime }+G(u)a(u)-G(0)a(0)-{\int }_0^u{G}^{\prime }a$. Use weak convergence of bounded-variation integrators for the first term and integration by parts for the remaining terms. No derivatives of the limiting continuous directions are required.

A map is Fréchet differentiable at $x$ if it admits a bounded linear first-order approximation: $f(x+h)=f(x)+Ah+o(\Vert h\Vert )$ for a continuous linear map $A$.

On the open set of invertible square matrices, the inversion map is differentiable and