Hadamard differentiability (${\dot{\Phi }}_s(a)$)

A map on a subset of a normed vector space is Hadamard differentiable at $s$ if its directional difference quotients converge to a continuous linear map uniformly along all convergent sequences of directions: $t_j\to 0$, $a_j\to a$, and $s+t_j{a}_j$ in the domain imply $(\Phi (s+t_j{a}_j)-\Phi (s))/t_j\to {\dot{\Phi }}_s(a)$. Tangential differentiability restricts the limit directions to a specified subspace. This is the regularity used by the functional delta method.

For a Hadamard differentiable functional with continuous linear derivative $L$, define $R_t(h)=(\Phi (s+th)-\Phi (s))/t-L(h)$. For every compact $K$ and positive $ϵ$, some positive $\delta ,a$ make $|R_t(h)|<ϵ$ whenever $\mathrm{dist}(h,K)<\delta$ and $0<|t|<a$. If this failed, choose $t_j\to 0$, directions within $1/j$ of $K$ and remainders at least $ϵ$. Compactness gives a convergent subsequence of nearby points of $K$, hence convergent directions. The derivative definition and continuity of $L$ give remainder tending to zero, a contradiction. With a tight weak limit, the Portmanteau theorem makes random directions lie near a suitable compact set with arbitrarily large probability. Thus this lemma proves the functional delta method without assuming separability of the entire normed space or uniform differentiability on its whole unit ball.

On bounded continuously differentiable functions with uniformly bounded total variation of a function, the map $T$ is Hadamard differentiable in the supremum norm with derivative ${\dot{T}}_{F,G}(a,b)={\int }_0^{u}b{F}^{\prime }+G(u)a(u)-G(0)a(0)-{\int }_0^u{G}^{\prime }a$. Use weak convergence of bounded-variation integrators for the first term and integration by parts for the remaining terms. No derivatives of the limiting continuous directions are required.