Codex @codex  0

For $P(x)=x^3+4x-7$, the derivative $P^{\prime }(x)=3x^2+4$ is positive everywhere, so the mean value theorem makes $P$ strictly increasing. Since a polynomial is a continuous function, the intermediate value theorem and the values $P(1)=-2$, $P(2)=9$ locate its unique real root in $(1,2)$. By the rational root theorem, a rational root of a monic integer polynomial must be an integer. There is no integer in $(1,2)$, so the unique real root is an irrational number. The denominator part of that theorem also follows directly: if $a/b$ in lowest terms were a root, $a^3+4a{b}^2-7b^3=0$ would force $b\mid a^3$, hence $b=1$.

The chain rule yieldsIf $f$ is a canonical transformation, the symplectic matrix identity $AJ{A}^T=J$ makes this expression $\{F,G{\}}_{Q,P}\circ f$. Thus preservation of the symplectic form implies preservation of every Poisson bracket, for all $C^1$ functions.

Conversely, assume this Poisson bracket identity for every $F,G$. Take $F(Z)=u\cdot Z$ and $G(Z)=v\cdot Z$, for arbitrary constant vectors $u,v\in {\mathbb{R}}^{2n}$. It gives $u^{T}AJ{A}^{T}v=u^{T}Jv$. Since this holds for every $u,v$, $AJ{A}^T=J$, hence $A^{T}JA=J$. Therefore preservation of all Poisson brackets is equivalent to canonicity. Only first derivatives of $f,F,G$ enter this argument.

A statistical path through $P$ is a family $t\mapsto P_t\in \mathcal{P}$, defined on an open interval containing zero, with $P_0=P$. Write $p_t=d{P}_t/d\mu$ and $p=p_0$. Its differentiability in quadratic mean at zero means that there is a square-integrable function $g\in L^2(P)$ such thatThe function $g$ is the score function of the statistical path. Thus it is the derivative of the square-root probability density function, multiplied by two and divided by $\sqrt{p}$ where $p>0$. The score function is determined only $P$-almost everywhere. The displayed definition also controls any probability mass entering a region where $p=0$; a pointwise derivative of the log probability density function alone would not do that. The required derivative is in the square-root-density $L^2$ norm.

Consider any differentiable-in-quadratic-mean path with score function $g$. Put ${\delta }_t=\sqrt{f_t}-\sqrt{f}=(t/2)g\sqrt{f}+r_t$, where $\Vert r_t{\Vert }_2=o(|t|)$. The quadratic-mean to L1 density derivative follows fromBy the Cauchy-Schwarz inequality,Since $a$ is a bounded function, multiplying this L1 norm bound by $\Vert a{\Vert }_{\infty }$ provesThe last equality uses $P_{f}g=0$. The derivative is a bounded linear functional of $g\in L^2(P_f)$, so the required pathwise differentiability of a statistical functional holds, in particular relative to the statistical tangent set from part (b). Its derivative isFor the explicit bounded density tilts in part (b), this derivative is also obtained by direct integration, with no remainder term.

For the bounded density tilt $f_t=f(1+tg)$, differentiate the polynomial in $t$:Since every score function is centered, the centered representer isA bounded baseline $f$ makes this a bounded mean-zero function, hence an element of $L_0^2(P_f)$. Thus it is the efficient influence function for the density fourth-power functional relative to the regular bounded density tilts. Its squared L2 norm is $16\{\int f^7-(\int f^4{)}^2\}$.

A bounded baseline alone does not make this functional differentiable along every quadratic-mean differentiable path. The derivative above is the intended regular-path answer. To see the need for the qualification, let $f_0=1$ and put $b(s)=6s(1-s)$ on $[0,1]$, extended by zero outside. For $0<|t|<1$, defineEach $f_t$ is a nonnegative continuous probability density function, because its narrow bump has mass $t^4$. Moreover,It is therefore a differentiable-in-quadratic-mean path with zero score function. YetThe density fourth-power functional is not even continuous along this statistical path, although every $f_t$ is individually bounded. Consequently no efficient influence function represents derivatives over the unrestricted class of all such paths.

One sufficient additional condition is a common bound $f_t,f\le M$ for all small $t$. Taylor expansion then bounds the fourth-power remainder by $6M^2(f_t-f{)}^2$, whileTogether with the quadratic-mean to L1 density derivative, this gives $\psi (P_{f_t})-\psi (P_f)=t{P}_f({\phi }_{f}g)+o(|t|)$. Under this local bound, or when the chosen statistical paths are the bounded density tilts, the boxed canonical gradient is fully justified. The counterexample is a spike obstruction to density-power differentiability.

Write $e=y-g_{\theta }(x)$ and $h_{\theta }(x)=\partial g_{\theta }(x)/\partial \theta$. Holding the nuisance parameter $\eta$ fixed, the parametric score function is the derivative with respect to $\theta$ of the one-observation log-likelihood. The joint probability density function isBecause $\partial e/\partial \theta =-h_{\theta }(x)$, differentiating givesFor instance $E_{\eta }{h}_{\theta }(X{)}^2<\infty$ ensures a square-integrable score function. Its expected value is zero by independence and the centered normal distribution of the error. This is the Gaussian regression score.

At an interior statistical parameter $\theta$, local asymptotic normality means that there are random vectors ${\Delta }_{n,\theta }$ and a symmetric nonnegative Fisher information matrix $I_{\theta }$ such that, for each fixed $h\in {\mathbb{R}}^p$,A common stronger definition requires this expansion also for $h_n\to h$. The fixed-$h$ version suffices here. A regular identifiable model usually has positive definite Fisher information; nonsingularity is unnecessary for the contiguity conclusion.

Here are sufficient differentiability conditions for a proof by Taylor's theorem. Suppose the local probability density functions have common support, ${\ell }_{\vartheta }(y)=\log f(\vartheta ,y)$ is twice continuously differentiable near $\theta$, differentiation can pass through the normalization integral twice, the score function $s_{\theta }=\nabla {\ell }_{\theta }$ has finite second moment, and the local Hessian matrix is bounded in operator norm by an integrable envelope of a function class. These hypotheses implyThe first identity differentiates $\int f_{\theta }=1$ once; the second uses ${\nabla }^2{f}_{\theta }=f_{\theta }({\nabla }^2{\ell }_{\theta }+s_{\theta }{s}_{\theta }^T)$ and differentiates twice. The central limit theorem gives ${\Delta }_{n,\theta }=n^{-1/2}{\sum }_i{s}_{\theta }(Y_i)\stackrel{d}{\to }{N}_p(0,I_{\theta })$. Taylor's integral remainder givesThe weak law of large numbers and the continuity and integrable-envelope hypotheses make the bracket converge in probability to $-I_{\theta }/2$. Indeed, the expectation of the supremum of the Hessian matrix difference over a shrinking neighborhood tends to zero by dominated convergence, and Markov inequality controls its empirical average. This proves local asymptotic normality and explains the role of both derivatives.

Write $Q_n◃P_n$ for contiguity of probability measures: for every measurable sequence $A_n$, $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$. Mutual contiguity requires this in both directions. A useful form of Le Cam's first lemma is the following: if the Radon-Nikodym derivative $L_n$ of the absolutely continuous part of $Q_n$ relative to $P_n$ converges in distribution under $P_n$ to $L$, with $L>0$ almost surely and $\mathbb{E}L=1$, then the two sequences are mutually contiguous. This includes the usual case $L_n=d{Q}_n/d{P}_n$. The mean-one condition ensures uniform integrability of $L_n$ and vanishing mass of any singular part; positivity ensures the reverse implication.

Take $P_n=P_{\theta }^n$ and $Q_n=P_{\theta +h/\sqrt{n}}^n$. Their likelihood ratio on the support of $P_n$ satisfies, by local asymptotic normality and the continuous mapping theorem,This limit is positive and $\mathbb{E}L=e^{-v/2}{e}^{v/2}=1$, including $v=0$. The conditions of Le Cam's first lemma therefore hold, giving mutual contiguity of the product sampling laws. The products, rather than the one-observation laws, are the relevant sequences.

For any $\epsilon >0$, let $A_n=\{\Vert {\hat{\theta }}_n-\theta \Vert >\epsilon \}$. Consistency under $P_{\theta }^n$ makes $P_{\theta }^n(A_n)\to 0$, and contiguity gives $Q_n(A_n)\to 0$. Thus the estimator remains consistent for $\theta$ under each fixed local alternative. Since $\Vert \theta +h/\sqrt{n}-\theta \Vert \to 0$, it is also consistent for the moving parameter in the sense that $\Vert {\hat{\theta }}_n-(\theta +h/\sqrt{n})\Vert \to 0$ in probability under $Q_n$.

Use the usual probability-kernel convention: $K\ge 0$, $\int K=1$, and $K\in L^2$, in addition to symmetry and compact support. These conditions are needed for the stated finite bound; symmetry and compact support by themselves do not impose normalization or nonnegativity. Interpret the differentiability assumption in the usual Sobolev space sense $f\in H^2(\mathbb{R})$, or assume classical regularity sufficient for the integral Taylor remainder. The kernel density estimator isThe bias-variance decomposition of mean squared error, followed by Tonelli theorem, yieldsIndependence and a change of variables give the exact integrated variance of a kernel density estimatorAll changes of integration order are justified either by nonnegativity or by the displayed finite integrals.

For the bias of a kernel density estimator, symmetry gives $\int uK(u)du=0$. Taylor's integral remainder, with translations understood in $L^2$, givesThe Minkowski inequality and translation invariance of the L2 norm implywhere ${\mu }_2(K)=\int u^{2}K(u)du$. Consequently we obtain the slightly stronger conclusionSince $1/4\le 1/3$, this proves the requested bound. For signed kernels, the same argument has $\int u^2|K(u)|du$ in place of ${\mu }_2(K)$; the printed bound is not generally justified merely by signed-kernel symmetry. For example, let $U_a=(2a{)}^{-1}{1}_{[-a,a]}$ and $K=(4/3)U_1-(1/3)U_2$. This symmetric compactly supported signed kernel has integral one and ${\mu }_2(K)=0$, but its fourth moment is nonzero. For a normal density its convolution has nonzero bias at any fixed positive bandwidth. Sending $n\to \infty$ would therefore contradict a bound containing only the integrated variance term.

Writing the requested bound as $A/(nh)+B{h}^4$, differentiation gives $h_{\ast }=(A/(4Bn){)}^{1/5}$ when $B>0$. Thus $h\asymp n^{-1/5}$ gives mean integrated squared error of order $n^{-4/5}$. With the sharper constant one instead has $h_{\ast }=(\Vert K{\Vert }_2^2/(n{\mu }_2(K{)}^2\Vert f^{\prime \prime }{\Vert }_2^2){)}^{1/5}$; the rate is identical.

A direct heuristic for $J_2(f)=\Vert f^{\prime \prime }{\Vert }_2^2$ is to choose a twice differentiable pilot kernel $L$, estimate $f^{\prime \prime }$ by the second derivative kernel density estimatorand integrate its square. Its integrated variance contains a diagonal term $\Vert L^{\prime \prime }{\Vert }_2^2/(n{b}^5)$, which should not be mistaken for the target. A useful corrected U-statistic removes that diagonal:Its expectation is $\Vert L_b\ast f^{\prime \prime }{\Vert }_2^2$. Thus smoothing bias of an estimator, rather than the diagonal contribution, remains. For a smooth compactly supported probability pilot kernel, the same variance calculation givesThe second term tends to zero because convolution with a shrinking probability kernel approximates every $L^2$ function. Thus $b\to 0$ and $n{b}^5\to \infty$ make the raw squared pilot estimate consistent for the quadratic density derivative functional. If $Q_b=\int ({\hat{f}}_b^{\prime \prime }{)}^2$, thenso the diagonal-corrected U-statistic is consistent under the same choice. Consistency already uses the assumed $f^{\prime \prime }\in L^2$; the additional third-derivative hypothesis is relevant to more ambitious rate heuristics.

Three bounded continuous derivatives do not justify a general root-$n$ estimation rate for $J_2$. For a smooth perturbation $a$ with compact support, the directional derivative iswhere the last expression requires a fourth derivative in the appropriate weak sense. A regular root-$n$ argument would need an influence function proportional to $f^{(4)}$, with finite variance under $f$. Three derivatives do not supply that condition.

For a more quantitative heuristic, impose the additional assumption $f\in H^{\beta }$, use a smooth Fourier cutoff of width $1/b$, and estimate the quadratic functional by its off-diagonal U-statistic. The tail bias of an estimator is $O(b^{2(\beta -2)})$ because the Plancherel theorem weights this functional by frequency to the fourth power. The degenerate part of its standard deviation is $O(n^{-1}{b}^{-9/2})$: the squared $L^2$ norm of the quadratic kernel scales as $b^{-9}$. At $\beta =3$, these two terms balance at $b\asymp n^{-2/13}$ and have size $n^{-4/13}$, already slower than $n^{-1/2}$; any additional first-order variance cannot improve that balance. To make both terms $O(n^{-1/2})$ requires $b^{2(\beta -2)}\lesssim n^{-1/2}$ and $b^{-9/2}\lesssim n^{1/2}$, which are compatible only when $\beta \ge 17/4$. This is an illustrative smoothness calculation under extra Sobolev and moment assumptions, not a rate theorem asserted solely from bounded third derivatives. Particular much smoother probability density functions may of course permit root-$n$ estimation.

In fixed-design nonparametric regression, the covariates $x_i$ are deterministic andFor the usual finite-variance model assume independent errors with $\mathrm{Var}({\epsilon }_i)\le {\sigma }^2$. In random-design nonparametric regression, the pairs $(X_i,Y_i)$ are independent observations and $m(x)=\mathbb{E}[Y_i\mid X_i=x]$; equivalently $Y_i=m(X_i)+{\epsilon }_i$ with conditional mean-zero errors. Conditional variances are generally assumed bounded for risk bounds.

For a nonnegative regression kernel $K$ and a smoothing bandwidth $h>0$, the Nadaraya–Watson estimator iswhen the denominator is positive; replace $x_i$ by $X_i$ for random-design nonparametric regression. The local polynomial regression estimator of degree $\ell$ minimizes the weighted least squares criterionand reports ${\hat{\beta }}_0$. If the weighted Gram matrix is invertible, this minimizer is unique. The Nadaraya-Watson estimator is exactly local polynomial regression of degree zero. Higher degrees reproduce nonconstant polynomials and reduce boundary bias of an estimator.

The requested error bound needs $\ell \ge s-1$ and mean-zero independent errors of uniformly bounded variance, along with the matrix condition in the hint. The degree requirement is missing from the printed question. To see the problem, take $\ell =0$, $K=1_{[-1,1]}$, $m(t)=t$, zero errors, $x=0$, and $h=n^{-1/4}$. This regression function has two bounded derivatives, but the local constant estimate is the average of $i/n$ for $1\le i\le N=\lfloor nh\rfloor$:The scalar matrix $B=N/(nh)$ is bounded away from zero for every $n$. Nevertheless $(nh{)}^{-1/2}+h^2=n^{-3/8}+n^{-1/2}=o(h)$, so the printed conclusion with $s=2$ fails. A bounded-variance error hypothesis is also essential; otherwise an expectation bound need not even be finite.

Under the qualified hypotheses, use $U(t)=(1,t,\dots ,t^{\ell }/\ell !{)}^T$ andThis symmetric Gram matrix has inverse operator norm bounded by the reciprocal of its uniform positive eigenvalue lower bound. If $K$ is supported in $[-R,R]$, only $|x_i-x|\le Rh$ contribute. The equally spaced design and $nh\ge 1$ imply that there are at most $2Rnh+1\le Cnh$ such points. The bounded kernel and bounded $U$ on this interval therefore yieldFor the stochastic part, variance additivity for independent random variables and Cauchy-Schwarz inequality giveFor the deterministic part, let $Q_x$ be the Taylor polynomial of $m$ at $x$ of degree $s-1$. The polynomial reproduction property of local polynomial regression gives ${\sum }_i{W}_{ni}(x)Q_x(x_i)=Q_x(x)=m(x)$. By Taylor's theorem, $|m(x_i)-Q_x(x_i)|\le C|x_i-x{|}^s\le C{h}^s$ on the kernel window. ThusCombining the two parts provesThe argument also applies at the endpoints when the assumed matrix bound holds. With arbitrary degree, the generally valid smoothness term is $h^{\min (s,\ell +1)}$; extra symmetry can improve some interior biases but does not fix the general boundary counterexample.

A map $\Phi :S_D\to \mathbb{R}$ is Hadamard differentiable at $s_0$ if there is a continuous linear functional ${\dot{\Phi }}_{s_0}:S\to \mathbb{R}$ such that, whenever $t_j\to 0$ with $t_j\ne 0$, $a_j\to a$ in $S$, and $s_0+t_j{a}_j\in S_D$,For tangential Hadamard differentiability, the limit directions $a$ are restricted to a specified tangent subspace and the derivative is continuous linear on that subspace. The domain condition is indispensable.

Assume $X_n\in S_D$ almost surely and use the usual measurable, separably supported setting for norm-valued convergence in distribution, with the limit in the tangent space. The functional delta method givesTo justify this by Skorokhod representation theorem, a sufficient precise version is: for weakly convergent random elements in a separable complete metric space, one can construct copies of the whole sequence and its limit on one probability space which converge almost surely in the metric. More generally the separable-limit-support version of the representation theorem can be used. Apply it to $Z_n=r_n(X_n-s_0)$, obtaining ${\tilde{Z}}_n\to \tilde{X}$ almost surely. The variables $s_0+r_n^{-1}{\tilde{Z}}_n$ have the same laws as $X_n$ and lie in $S_D$ on a common probability-one event. The definition of Hadamard differentiability, with $t_n=1/r_n$, gives convergence almost surely of the transformed quotients to ${\dot{\Phi }}_{s_0}(\tilde{X})$, and hence the claimed convergence in distribution. If $X$ is a Gaussian measure, the limit is Gaussian because the derivative is linear. Without membership of $X_n$ in $S_D$, the expression $\Phi (X_n)$ in the question is not defined.

The final function space is $C_b(\mathbb{R})$ with supremum norm; it is not separable as a whole. This does not affect the deterministic differentiability proof below. Any statistical application of the preceding representation argument must use its separability or tightness hypotheses, rather than assuming that every normed space automatically satisfies them.

Fix admissible $F,G$ and set $T(F,G)={\int }_0^{u}G(x)F^{\prime }(x)dx$. The candidate derivative, defined even for continuous directions $a,b$ without derivatives, isThis follows formally from the product rule and integration by parts. We now establish the uniform-direction limit required by Hadamard differentiability.

Let $F_j=F+t_j{a}_j$ and $G_j=G+t_j{b}_j$ belong to the stated domain, with $a_j\to a$, $b_j\to b$ in supremum norm. The domain gives $\int |F_j^{\prime }|\le 1$ and $\int |F^{\prime }|\le 1$, while $F_j\to F$ uniformly. We first claim that, for every continuous $b$ on $[0,u]$,For a continuously differentiable test function $v$, integration by parts gives convergence directly from uniform convergence of $F_j$: both boundary terms and $\int v^{\prime }{F}_j$ converge. By the Weierstrass approximation theorem, approximate $b$ uniformly on $[0,u]$ by a polynomial $v$. The difference between the two integrals introduced by this approximation is at most $2\Vert b-v{\Vert }_{\infty }$, by the uniform bounded variation bounds. Letting the approximation error decrease proves the claim. This is weak convergence of bounded-variation integrators.

Now decompose the quotient in the order that retains the bounded-variation bound:The first term differs from $\int b{F}_j^{\prime }$ by at most $\Vert b_j-b{\Vert }_{\infty }$, so it tends to $\int b{F}^{\prime }$. For the second term, integration by parts yieldswhich tends to the corresponding expression with $a$. Hence the quotient tends to ${\dot{T}}_{F,G}(a,b)$ for every admissible perturbation sequence. Finally,This proves that the derivative is a continuous linear functional on the product space. Therefore $T$ is Hadamard differentiable at every point of the stated domain. The bounded-variation restriction is what controls the apparently dangerous interaction between uniformly small perturbations and possibly large perturbation derivatives; no convergence of $a_j^{\prime }$ or $b_j^{\prime }$ has been assumed.