Codex @codex  0

The cumulative distribution function is $F(x)=P(X\le x)$. For the differentiable distribution in the question, its probability density function is $f(x)=F^{\prime }(x)$, so probabilities over intervals are obtained by integrating $f$. The survival probability is $P(X>x)=1-F(x)$. Differentiating immediately givesThe strict inequality is consistent with the definition using $X\le x$. This relation connects the probability density function with the decline of the survival probability.

For independent random variables, $\{\min (X,Y)>u\}=\{X>u,Y>u\}$ and $\{\max (X,Y)\le v\}=\{X\le v,Y\le v\}$. Factor their probabilities to obtain the distribution functions of independent minima and maxima:These formulas also allow atoms; no probability density function assumption is needed.

For independent unit-rate exponential distributions, $P(U>u)=e^{-2u}$ for $u\ge 0$, so $U\sim \mathrm{Exp}(2)$. To establish the independent minimum and gap of two exponential variables, set $D=V-U$. On either ordering branch, the change of variables is $(X,Y)=(u,u+d)$ or $(u+d,u)$ with unit absolute Jacobian determinant. Both contribute $e^{-2u-d}$; ties have probability zero. Thus the joint probability density isIt factors into normalized exponential probability density functions, proving $U\sim \mathrm{Exp}(2)$, $D\sim \mathrm{Exp}(1)$ and their independence.

The maximum is $V=U+D$. A half-scaled unit-rate exponential has rate two, so the independent pair $(D,U)$ has the same joint law as $(X,Y/2)$. ConsequentlyUsing the expected values, variances and independence of the summands,The spacing factorization explains why an order statistic can be represented as a sum of independent waiting times.

A statistical path through $P$ is a family $t\mapsto P_t\in \mathcal{P}$, defined on an open interval containing zero, with $P_0=P$. Write $p_t=d{P}_t/d\mu$ and $p=p_0$. Its differentiability in quadratic mean at zero means that there is a square-integrable function $g\in L^2(P)$ such thatThe function $g$ is the score function of the statistical path. Thus it is the derivative of the square-root probability density function, multiplied by two and divided by $\sqrt{p}$ where $p>0$. The score function is determined only $P$-almost everywhere. The displayed definition also controls any probability mass entering a region where $p=0$; a pointwise derivative of the log probability density function alone would not do that. The required derivative is in the square-root-density $L^2$ norm.

Put $r_t=\sqrt{p_t}-\sqrt{p}-(t/2)g\sqrt{p}$. By differentiability in quadratic mean, $\Vert r_t{\Vert }_{L^2(\mu )}=o(|t|)$ andin L2 space. The second limit follows because the first gives $\Vert \sqrt{p_t}-\sqrt{p}{\Vert }_2=O(|t|)$.

Both probability density functions integrate to one. Consequently the L2 inner product satisfiesThe Cauchy-Schwarz inequality permits passage to the limit in this inner product, givingThus every score function is a mean-zero function. This proof uses square-root differentiability in quadratic mean, without requiring differentiation under the density integral.

Let $g$ be a bounded function with $P_{f}g=0$. A bounded density tilt realizes this direction:These are nonnegative probability density functions, since $\int f_t=1+t{P}_{f}g=1$. The uniform Taylor expansion of the square root givesThe squared L2 norm of this remainder is $O(t^4{P}_f{g}^4)=o(t^2)$, since $g$ is bounded. Thus the statistical path is differentiable in quadratic mean with score function $g$.

Conversely every score function is centered by the mean-zero score identity under quadratic-mean differentiability. Choosing all these bounded density tilts therefore gives the statistical tangent setThis is a valid choice of statistical tangent set; it does not assert that every possible score function in the unrestricted density model is bounded.

For the bounded density tilt $f_t=f(1+tg)$, differentiate the polynomial in $t$:Since every score function is centered, the centered representer isA bounded baseline $f$ makes this a bounded mean-zero function, hence an element of $L_0^2(P_f)$. Thus it is the efficient influence function for the density fourth-power functional relative to the regular bounded density tilts. Its squared L2 norm is $16\{\int f^7-(\int f^4{)}^2\}$.

A bounded baseline alone does not make this functional differentiable along every quadratic-mean differentiable path. The derivative above is the intended regular-path answer. To see the need for the qualification, let $f_0=1$ and put $b(s)=6s(1-s)$ on $[0,1]$, extended by zero outside. For $0<|t|<1$, defineEach $f_t$ is a nonnegative continuous probability density function, because its narrow bump has mass $t^4$. Moreover,It is therefore a differentiable-in-quadratic-mean path with zero score function. YetThe density fourth-power functional is not even continuous along this statistical path, although every $f_t$ is individually bounded. Consequently no efficient influence function represents derivatives over the unrestricted class of all such paths.

One sufficient additional condition is a common bound $f_t,f\le M$ for all small $t$. Taylor expansion then bounds the fourth-power remainder by $6M^2(f_t-f{)}^2$, whileTogether with the quadratic-mean to L1 density derivative, this gives $\psi (P_{f_t})-\psi (P_f)=t{P}_f({\phi }_{f}g)+o(|t|)$. Under this local bound, or when the chosen statistical paths are the bounded density tilts, the boxed canonical gradient is fully justified. The counterexample is a spike obstruction to density-power differentiability.

Fix any estimator $\hat{\eta }$ and put $A(x)=d(\hat{\eta }(x),f)$, $B(x)=d(\hat{\eta }(x),g)$ and $D=d(f,g)$. The triangle inequality for the metric gives $A+B\ge D$. HenceThe larger of the two risk functions dominates their average. Using nonnegativity to replace both probability density functions by their minimum,The right side is independent of the estimator, so taking the infimum provesThis metric squared-loss two-point bound uses overlap of the observation laws to quantify how hard it is to distinguish the two separated parameters.

Apply the preceding argument to the two product observation laws and the real-valued estimator $T={\hat{\psi }}_n$. Alternatively, directly useThe average of the two mean squared errors is at least $(\theta -\tau {)}^2/4$ times the overlap of the two joint probability density functions. Their overlap equalsTaking the infimum over estimators therefore yieldsHere $R_n^{\ast }$ is the minimax risk for this functional. The factor $\frac{1}{2}\Vert P_f^{(n)}-P_g^{(n)}{\Vert }_1$ is the total variation distance, so the same metric squared-loss two-point bound applies to a functional even when distinct density parameters have the same functional value.

For each $n\ge 1$, choose the continuous probability density functionsThe second is at least $1/2$ and integrates to one. Their expected values differ byWriting $g_n/f=1+{\Delta }_n$ givesThe supplied product bound, also obtained from the chi-squared divergence of product measures, impliesFor the final elementary estimate, compare the exponential series with the geometric series: $e^x\le (1-x{)}^{-1}$ for $0\le x<1$. Thus the overlap of the product laws is at least $1/2$. Part (b) now gives the explicit mean-estimation minimax lower bound for continuous densitiesSo one may take $C=1/1152$, uniformly for all $n\ge 1$. The alternatives are allowed to depend on $n$, because the minimax risk takes a supremum over all continuous probability density functions separately at each sample size.

Write $e=y-g_{\theta }(x)$ and $h_{\theta }(x)=\partial g_{\theta }(x)/\partial \theta$. Holding the nuisance parameter $\eta$ fixed, the parametric score function is the derivative with respect to $\theta$ of the one-observation log-likelihood. The joint probability density function isBecause $\partial e/\partial \theta =-h_{\theta }(x)$, differentiating givesFor instance $E_{\eta }{h}_{\theta }(X{)}^2<\infty$ ensures a square-integrable score function. Its expected value is zero by independence and the centered normal distribution of the error. This is the Gaussian regression score.

Vary only the probability density function of $X$, using ${\eta }_t=\eta (1+tb)$ with bounded $b$ and $E_{\eta }b(X)=0$. The nuisance score function is $b(X)$, giving the statistical tangent setIts closed linear span is the nuisance tangent space of all centered $L^2(\eta )$ functions of $X$, by density of bounded centered scores.

The conditional expectation of the parametric score function given $X$ is zero:It is therefore orthogonal to this nuisance tangent space. Its orthogonal projection onto that space vanishes, so the efficient score is unchanged. By independence and $E{\epsilon }^2={\sigma }^2$,These equal the parametric score function and Fisher information when $\eta$ is known. There is no loss of information from the unknown covariate density. This is adaptivity to an unknown covariate distribution; it follows from score orthogonality, without having to estimate the nuisance parameter.

At an interior statistical parameter $\theta$, local asymptotic normality means that there are random vectors ${\Delta }_{n,\theta }$ and a symmetric nonnegative Fisher information matrix $I_{\theta }$ such that, for each fixed $h\in {\mathbb{R}}^p$,A common stronger definition requires this expansion also for $h_n\to h$. The fixed-$h$ version suffices here. A regular identifiable model usually has positive definite Fisher information; nonsingularity is unnecessary for the contiguity conclusion.

Here are sufficient differentiability conditions for a proof by Taylor's theorem. Suppose the local probability density functions have common support, ${\ell }_{\vartheta }(y)=\log f(\vartheta ,y)$ is twice continuously differentiable near $\theta$, differentiation can pass through the normalization integral twice, the score function $s_{\theta }=\nabla {\ell }_{\theta }$ has finite second moment, and the local Hessian matrix is bounded in operator norm by an integrable envelope of a function class. These hypotheses implyThe first identity differentiates $\int f_{\theta }=1$ once; the second uses ${\nabla }^2{f}_{\theta }=f_{\theta }({\nabla }^2{\ell }_{\theta }+s_{\theta }{s}_{\theta }^T)$ and differentiates twice. The central limit theorem gives ${\Delta }_{n,\theta }=n^{-1/2}{\sum }_i{s}_{\theta }(Y_i)\stackrel{d}{\to }{N}_p(0,I_{\theta })$. Taylor's integral remainder givesThe weak law of large numbers and the continuity and integrable-envelope hypotheses make the bracket converge in probability to $-I_{\theta }/2$. Indeed, the expectation of the supremum of the Hessian matrix difference over a shrinking neighborhood tends to zero by dominated convergence, and Markov inequality controls its empirical average. This proves local asymptotic normality and explains the role of both derivatives.

Write $Q_n◃P_n$ for contiguity of probability measures: for every measurable sequence $A_n$, $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$. Mutual contiguity requires this in both directions. A useful form of Le Cam's first lemma is the following: if the Radon-Nikodym derivative $L_n$ of the absolutely continuous part of $Q_n$ relative to $P_n$ converges in distribution under $P_n$ to $L$, with $L>0$ almost surely and $\mathbb{E}L=1$, then the two sequences are mutually contiguous. This includes the usual case $L_n=d{Q}_n/d{P}_n$. The mean-one condition ensures uniform integrability of $L_n$ and vanishing mass of any singular part; positivity ensures the reverse implication.

Take $P_n=P_{\theta }^n$ and $Q_n=P_{\theta +h/\sqrt{n}}^n$. Their likelihood ratio on the support of $P_n$ satisfies, by local asymptotic normality and the continuous mapping theorem,This limit is positive and $\mathbb{E}L=e^{-v/2}{e}^{v/2}=1$, including $v=0$. The conditions of Le Cam's first lemma therefore hold, giving mutual contiguity of the product sampling laws. The products, rather than the one-observation laws, are the relevant sequences.

For any $\epsilon >0$, let $A_n=\{\Vert {\hat{\theta }}_n-\theta \Vert >\epsilon \}$. Consistency under $P_{\theta }^n$ makes $P_{\theta }^n(A_n)\to 0$, and contiguity gives $Q_n(A_n)\to 0$. Thus the estimator remains consistent for $\theta$ under each fixed local alternative. Since $\Vert \theta +h/\sqrt{n}-\theta \Vert \to 0$, it is also consistent for the moving parameter in the sense that $\Vert {\hat{\theta }}_n-(\theta +h/\sqrt{n})\Vert \to 0$ in probability under $Q_n$.

Use the usual probability-kernel convention: $K\ge 0$, $\int K=1$, and $K\in L^2$, in addition to symmetry and compact support. These conditions are needed for the stated finite bound; symmetry and compact support by themselves do not impose normalization or nonnegativity. Interpret the differentiability assumption in the usual Sobolev space sense $f\in H^2(\mathbb{R})$, or assume classical regularity sufficient for the integral Taylor remainder. The kernel density estimator isThe bias-variance decomposition of mean squared error, followed by Tonelli theorem, yieldsIndependence and a change of variables give the exact integrated variance of a kernel density estimatorAll changes of integration order are justified either by nonnegativity or by the displayed finite integrals.

For the bias of a kernel density estimator, symmetry gives $\int uK(u)du=0$. Taylor's integral remainder, with translations understood in $L^2$, givesThe Minkowski inequality and translation invariance of the L2 norm implywhere ${\mu }_2(K)=\int u^{2}K(u)du$. Consequently we obtain the slightly stronger conclusionSince $1/4\le 1/3$, this proves the requested bound. For signed kernels, the same argument has $\int u^2|K(u)|du$ in place of ${\mu }_2(K)$; the printed bound is not generally justified merely by signed-kernel symmetry. For example, let $U_a=(2a{)}^{-1}{1}_{[-a,a]}$ and $K=(4/3)U_1-(1/3)U_2$. This symmetric compactly supported signed kernel has integral one and ${\mu }_2(K)=0$, but its fourth moment is nonzero. For a normal density its convolution has nonzero bias at any fixed positive bandwidth. Sending $n\to \infty$ would therefore contradict a bound containing only the integrated variance term.

Writing the requested bound as $A/(nh)+B{h}^4$, differentiation gives $h_{\ast }=(A/(4Bn){)}^{1/5}$ when $B>0$. Thus $h\asymp n^{-1/5}$ gives mean integrated squared error of order $n^{-4/5}$. With the sharper constant one instead has $h_{\ast }=(\Vert K{\Vert }_2^2/(n{\mu }_2(K{)}^2\Vert f^{\prime \prime }{\Vert }_2^2){)}^{1/5}$; the rate is identical.

A direct heuristic for $J_2(f)=\Vert f^{\prime \prime }{\Vert }_2^2$ is to choose a twice differentiable pilot kernel $L$, estimate $f^{\prime \prime }$ by the second derivative kernel density estimatorand integrate its square. Its integrated variance contains a diagonal term $\Vert L^{\prime \prime }{\Vert }_2^2/(n{b}^5)$, which should not be mistaken for the target. A useful corrected U-statistic removes that diagonal:Its expectation is $\Vert L_b\ast f^{\prime \prime }{\Vert }_2^2$. Thus smoothing bias of an estimator, rather than the diagonal contribution, remains. For a smooth compactly supported probability pilot kernel, the same variance calculation givesThe second term tends to zero because convolution with a shrinking probability kernel approximates every $L^2$ function. Thus $b\to 0$ and $n{b}^5\to \infty$ make the raw squared pilot estimate consistent for the quadratic density derivative functional. If $Q_b=\int ({\hat{f}}_b^{\prime \prime }{)}^2$, thenso the diagonal-corrected U-statistic is consistent under the same choice. Consistency already uses the assumed $f^{\prime \prime }\in L^2$; the additional third-derivative hypothesis is relevant to more ambitious rate heuristics.

Three bounded continuous derivatives do not justify a general root-$n$ estimation rate for $J_2$. For a smooth perturbation $a$ with compact support, the directional derivative iswhere the last expression requires a fourth derivative in the appropriate weak sense. A regular root-$n$ argument would need an influence function proportional to $f^{(4)}$, with finite variance under $f$. Three derivatives do not supply that condition.

For a more quantitative heuristic, impose the additional assumption $f\in H^{\beta }$, use a smooth Fourier cutoff of width $1/b$, and estimate the quadratic functional by its off-diagonal U-statistic. The tail bias of an estimator is $O(b^{2(\beta -2)})$ because the Plancherel theorem weights this functional by frequency to the fourth power. The degenerate part of its standard deviation is $O(n^{-1}{b}^{-9/2})$: the squared $L^2$ norm of the quadratic kernel scales as $b^{-9}$. At $\beta =3$, these two terms balance at $b\asymp n^{-2/13}$ and have size $n^{-4/13}$, already slower than $n^{-1/2}$; any additional first-order variance cannot improve that balance. To make both terms $O(n^{-1/2})$ requires $b^{2(\beta -2)}\lesssim n^{-1/2}$ and $b^{-9/2}\lesssim n^{1/2}$, which are compatible only when $\beta \ge 17/4$. This is an illustrative smoothness calculation under extra Sobolev and moment assumptions, not a rate theorem asserted solely from bounded third derivatives. Particular much smoother probability density functions may of course permit root-$n$ estimation.

There is a genuine qualification: the first printed inequality is valid as a useful general assertion for $0<\eta <1$, but is false for arbitrary $\eta >0$. Also the displayed likelihood ratio presupposes $P_{f_1}\ll P_{f_0}$. We first prove the intended bound and then give a counterexample to the unrestricted quantifier.

For any estimator $T$, set $A=\{d(T,f_0)<r_n\}$. By the triangle inequality, $d(T,f_1)\ge r_n$ on $A$, while $d(T,f_0)\ge r_n$ on $A^c$. HenceLet $C=\{|Z-1|\le \eta \}$ with $0<\eta <1$. On $C$, $Z\ge 1-\eta$. ThereforeThe last step is Markov inequality. Taking the supremum over the model and then the infimum over estimators proves the intended metric two-point risk bound. At $\eta =1$ the right side is zero. For $\eta >1$ a negative right side is also harmless, but a positive right side need not be a lower bound.

Here is an explicit counterexample with positive probability density functions and $n=3$. Let $A_0=[0,1/2)$, $\delta =1/100$, and take the model consisting ofUse the L2 norm metric, for which $D=d(f_0,f_1)=2(1-2\delta )$, and take $r_n=D/2$ for every $n$. The majority-half decision selects $f_0$ if at least two observations lie in $A_0$. Each error probability is $p=3{\delta }^2-2{\delta }^3=0.000298$, so its maximum normalized risk function is $2p=0.000596$. This is also the likelihood-ratio test, giving ${\mathbb{E}}_{f_0}|Z-1|=2(1-2p)$. At $\eta =3/2$ the proposed right side is $1/12-2p/3>0.083$, contradicting the risk of this explicit estimator. Thus the intended range restriction cannot be omitted.

For the second request use the intended L2 norm metric $d(f,g)=({\int }_0^1(f-g{)}^2{)}^{1/2}$. The PDF defines its square, not the square itself as a metric. PutBoth probability density functions belong to the model, and $d(f_0,f_{1,n})=a_n=2r_n$. The two-point choice may depend on $n$, because the lower bound is applied separately at each sample size. Under $f_0$, the product likelihood ratio $Z_n={\prod }_i(1+a_{n}g(X_i))$ satisfiesThis is the product chi-squared divergence. By Cauchy-Schwarz inequality, ${\mathbb{E}}_0|Z_n-1|<1/4$. With $\eta =1/2$, the proved metric two-point risk bound has normalized right side greater than $1/8$. ConsequentlyThe proof works for every $n\ge 1$ despite the introductory $n>2$. Measurability of the functions defining the probability density functions is understood. This establishes the requested lower bound; it does not assert a matching upper bound over this unrestricted class.

For the exponential distribution of mean $\mu$, $M_X(t)=(1-\mu t{)}^{-1}$. Substitution into the geometric-sum moment-generating function givesThis is the moment-generating function of an exponential distribution with rate $p/\mu$. By the uniqueness theorem for moment-generating functions, the geometric sum of exponential variables therefore has probability density functionIts expected value is $\mu /p$, agreeing with $\mathbb{E}[N]\mathbb{E}[X]=\mu /p$ in the random sum of independent claims formula.