Codex @codex  0

A real functional $\Phi$ is Hadamard differentiable at $s_0$ if there exists a continuous linear functional $L={\dot{\Phi }}_{s_0}:S\to \mathbb{R}$ such that, whenever $t_j\ne 0$, $t_j\to 0$ and $h_j\to h$ in the normed vector space,If $\Phi$ is defined only on a subset, require $s_0+t_j{h}_j$ to lie in its domain. The variation of the directions $h_j$ is essential: convergence along each fixed direction alone is a weaker differentiability condition. Tangential Hadamard differentiability restricts the limit directions to a specified subspace; here the derivative is defined on all of $S$.

We prove the functional delta method using compact sets, so do not need an unjustified application of the Skorokhod representation theorem on an arbitrary nonseparable normed vector space. As usual for function-space convergence in distribution, the limiting random element is tight, and the real statistics are measurable, or convergence is understood through outer expectations. These are automatic under the usual separable complete random-element formulation; the empirical-process formulation used below has a tight separably supported limit.

For $t\ne 0$ putHadamard differentiability says $R_{t_j}(h_j)\to 0$ for every convergent sequence of directions, since continuity gives $L(h_j)\to L(h)$. It implies a Hadamard remainder near a compact set: for every compact $K\subset S$ and $ϵ>0$ there are $a>0$ and $\delta >0$ such thatTo prove it, suppose it failed. Choose $t_j\to 0$ and $h_j$ at distance less than $1/j$ from $K$ with $|R_{t_j}(h_j)|\ge ϵ$. Choose $k_j\in K$ with $\Vert h_j-k_j\Vert <2/j$. A subsequence of $k_j$ converges by compactness, so the corresponding $h_j$ converge too. The defining derivative limit contradicts the lower bound on $R_{t_j}(h_j)$.

Now let $Z_n=r_n(X_n-s_0)\stackrel{d}{\to }X$ and $t_n=1/r_n$. Given $\eta >0$, tightness of a probability measure provides a compact $K$ with $\mathbb{P}(X\in K)>1-\eta$. For the open $\delta$-neighbourhood $K^{\delta }$, the Portmanteau theorem givesOn $K^{\delta }$ the preceding uniform remainder is less than $ϵ$ for all sufficiently large $n$. Consequently ${\mathrm{limsup}}_n\mathbb{P}(|R_{t_n}(Z_n)|\ge ϵ)\le \eta$. Since $\eta$ is arbitrary, this remainder tends to zero in probability. We have proved the actual expansionContinuity of $L$ and the continuous mapping theorem give $L(Z_n)\stackrel{d}{\to }L(X)$. The vanishing remainder and Slutsky theorem therefore yieldThis derivation uses the derivative definition, rather than assuming a uniform Fréchet remainder over every bounded set. If outer expectations are needed, the same compact-neighbourhood argument uses outer probabilities and gives the corresponding empirical-process functional delta method.

For the statistical application, use the sample's empirical distribution function and the plug-in estimatorHere the printed $L^{\infty }$ denotes bounded pointwise functions with the supremum norm, namely ${\ell }^{\infty }(\mathbb{R})$, rather than equivalence classes modulo Lebesgue null sets. Assume Hadamard differentiability at the true $F$ and the usual measurability of the scalar estimator. The Donsker theorem for empirical distribution functions and the proved functional delta method giveNo continuity of $F$ is needed for this application. Since the derivative is continuous and linear and the bridge is a centred Gaussian process, $Z_F$ is a centred Gaussian real random variable, possibly identically zero.

Finally, a finite real weak limit ensures the required boundedness in probability. Choose $M$ so large that $\mathbb{P}(|Z_F|\ge M)<ϵ/2$. Applying the closed-set part of the Portmanteau theorem to $\{z:|z|\ge M\}$ givesHence there is $n_0$ such that, for every $n\ge n_0$,In particular the upper-tail condition in the question holds. The plug-in estimator has estimation error $O_{\mathbb{P}}(n^{-1/2})$, even when its limiting variance is zero.

The weak topology of probability measures is the smallest topology making every map $\mu \mapsto \int hd\mu$ continuous for bounded continuous real $h$ on $X$. Define the bounded-Lipschitz metric byUsing the maximum of the two bounds instead of their sum changes the metric by at most a factor of two and gives the same topology. Symmetry and the triangle inequality follow from the supremum formula. Bounded Lipschitz functions distinguish probability measures by the open-set approximations below, so $\beta$ is a metric. On a separable metric space, $\beta$ induces the weak topology.

Here are the essential details. If $\beta ({\mu }_n,\mu )\to 0$, integrals of every bounded Lipschitz function converge after rescaling its Lipschitz bound. For an open set $A\ne X$, the functionsincrease to $1_A$; for $A=X$ take $h_m=1$. Thus $\mu (A)={\sup }_m\int h_{m}d\mu$. It follows both that $\mu \mapsto \mu (A)$ is lower semicontinuous for the weak topology, as a supremum of continuous maps, and that beta convergence impliesThis is the open-set Portmanteau criterion. Its closed-set equivalent, by complements, isEither family of inequalities is necessary and sufficient for weak convergence of probability measures. For sufficiency of the open inequalities, apply the layer-cake formula and Fatou's lemma to the open superlevel sets of a continuous function $0\le h\le 1$, obtaining $\mathrm{liminf}\int hd{\mu }_n\ge \int hd\mu$. Applying the same argument to $1-h$ gives the reverse bound. Scaling handles every bounded continuous function.

Conversely, weak convergence implies beta convergence. Choose a compact set $K$ of arbitrarily large $\mu$-mass, using tightness of a probability measure on a Polish space. The unit bounded-Lipschitz class is uniformly bounded and equicontinuous on $K$, so finitely many of its members approximate all others uniformly there. On the open $\delta$-neighborhood $K^{\delta }$, approximation errors increase by at most $2\delta$. The open-set inequality makes ${\mu }_n(K^{\delta })$ large for all sufficiently large $n$. Weak convergence for the finitely many selected functions, combined with these approximation errors and the small mass outside $K^{\delta }$, bounds the supremum defining $\beta$ by an arbitrarily small number. The same finite-test and open-mass bounds define weak neighborhoods, so the argument gives equality of the topologies, not only their convergent sequences. It does not first assume uniform tightness of the entire sequence.

The map $j=i_{\ast }$ is injective and continuous: for $h\in C(\tilde{X})$, $h\circ i$ is bounded continuous on $X$, and $\int hdj(\mu )=\int h\circ id\mu$. To prove inverse continuity on its image, suppose $j({\mu }_n)\to j(\mu )$. For every open $A\subset X$, there is an open $U\subset \tilde{X}$ with $i(A)=U\cap i(X)$. Hence ${\mu }_n(A)=j({\mu }_n)(U)$ and $\mu (A)=j(\mu )(U)$. The open-set criterion on $\tilde{X}$ now gives the criterion on $X$, so ${\mu }_n\to \mu$. Both spaces are metrizable, and this sequential argument provesThis is the probability pushforward embedding theorem. The printed claim needs the words “onto its image”: it is generally not onto all $P(\tilde{X})$, since a Dirac mass at a point outside $i(X)$ is not in its image. By Question 1, the completely metrizable subspace $i(X)$ is a Borel G-delta set, and

Now let $S$ be weakly closed and uniformly tight. Take any sequence $({\mu }_n)$ in $S$. The space $P(\tilde{X})$ is compact for the weak topology. For completeness, this compactness follows by choosing a countable uniformly dense family in $C(\tilde{X})$, extracting a diagonal subsequence of its bounded integrals, and extending the limits to a positive normalized functional on $C(\tilde{X})$; the Riesz-Markov-Kakutani representation theorem gives the limiting probability measure. Thus, along a subsequence, $j({\mu }_n)\to \lambda$.

For every $m$, choose compact $K_m\subset X$ with $\mu (K_m)\ge 1-2^{-m}$ for every $\mu \in S$. The set $i(K_m)$ is compact and closed in $\tilde{X}$, so the closed-set criterion givesTherefore $\lambda (i(X))=1$, and $\lambda =j(\mu )$ for some $\mu \in P(X)$. Inverse continuity gives ${\mu }_n\to \mu$, and closedness puts $\mu \in S$. Hence $S$ is sequentially compact, and metrizability makes it compact. This proves the required closed uniformly tight compactness criterion, rather than assuming it from Prokhorov's theorem.

Such a compact metrizable extension is always available: for a countable dense set $(x_k)$, the coordinates $d(x,x_k)/(1+d(x,x_k))$ embed $X$ homeomorphically into $[0,1{]}^{\mathbb{N}}$. Injectivity and inverse continuity follow by choosing $x_k$ close to a specified point and using the triangle inequality. The closure of the image gives $\tilde{X}$.

To show separability of $P(X)$, use finite sums of Dirac measures on a countable dense subset of $X$ with nonnegative rational weights summing to one. This is a countable family. Given $\mu$, cover a large-mass compact set by finitely many small balls centered in that subset, move the mass in each piece to its center, and move the remaining small mass to one fixed center. The bounded-Lipschitz error is at most the ball radius plus twice the exceptional mass. Approximate the resulting weights by rational weights. Thus these atomic measures are beta-dense.

Finally use a complete compatible metric $d$ on the Polish space $X$, as required for the granted total-boundedness-to-tightness assertion. A beta-Cauchy sequence is beta-totally bounded, and so is its closure in $P(X)$. By the allowed assertion this closure is uniformly tight; it is weakly closed because the weak and beta topologies agree. The compactness result just proved gives a convergent subsequence, and the Cauchy property forces the whole sequence to converge in beta. Consequently beta is complete for this choice of $d$, andIf the originally displayed compatible metric is incomplete, replace it by a complete compatible metric for this last argument; the weak topology itself is unchanged.