Codex @codex  0

A statistical path through $P$ is a family $t\mapsto P_t\in \mathcal{P}$, defined on an open interval containing zero, with $P_0=P$. Write $p_t=d{P}_t/d\mu$ and $p=p_0$. Its differentiability in quadratic mean at zero means that there is a square-integrable function $g\in L^2(P)$ such thatThe function $g$ is the score function of the statistical path. Thus it is the derivative of the square-root probability density function, multiplied by two and divided by $\sqrt{p}$ where $p>0$. The score function is determined only $P$-almost everywhere. The displayed definition also controls any probability mass entering a region where $p=0$; a pointwise derivative of the log probability density function alone would not do that. The required derivative is in the square-root-density $L^2$ norm.

Put $r_t=\sqrt{p_t}-\sqrt{p}-(t/2)g\sqrt{p}$. By differentiability in quadratic mean, $\Vert r_t{\Vert }_{L^2(\mu )}=o(|t|)$ andin L2 space. The second limit follows because the first gives $\Vert \sqrt{p_t}-\sqrt{p}{\Vert }_2=O(|t|)$.

Both probability density functions integrate to one. Consequently the L2 inner product satisfiesThe Cauchy-Schwarz inequality permits passage to the limit in this inner product, givingThus every score function is a mean-zero function. This proof uses square-root differentiability in quadratic mean, without requiring differentiation under the density integral.

Fix $P=P_{\theta ,\eta }$. Let $\mathcal{N}$ be the nuisance tangent space: the closed linear span in $L^2(P)$ of score functions of statistical paths that vary only the nuisance parameter $\eta$. By the preceding argument, $\mathcal{N}$ is contained in the mean-zero L2 space $L_0^2(P)$.

Let ${\Pi }_{\mathcal{N}}$ denote orthogonal projection onto this closed subspace of a Hilbert space. The efficient score and scalar efficient information areThe efficient score is the component of the parametric score function that cannot be reproduced by changing the nuisance parameter. The efficient information is its squared L2 norm; it can be zero, so positivity must not be assumed in the definition.

The map $h\mapsto Ph$ is a bounded linear functional on L2 space, since $|Ph|\le \Vert h{\Vert }_2$ by the Cauchy-Schwarz inequality. Its kernel, the mean-zero L2 space, is therefore closed. Every nuisance score function and the parametric score function $\dot{\ell }$ are centered, so both ${\Pi }_{\mathcal{N}}\dot{\ell }$ and $\tilde{\ell }$ are centered. HenceMoreover, the efficient score belongs to the orthogonal complement of the nuisance tangent space. Writing $\dot{\ell }=\tilde{\ell }+{\Pi }_{\mathcal{N}}\dot{\ell }$ givesThis is the efficient-score projection identity; it remains valid when the efficient information is zero.

Choose a collection of statistical paths through $P_f$ that are differentiable in quadratic mean. A statistical tangent set at $P_f$ is the set of their score functions. In particular each member belongs to $L_0^2(P_f)$, by the mean-zero score identity under quadratic-mean differentiability.

A statistical tangent set records which first-order directions the chosen statistical paths can realize. Its closed linear span in L2 space is the statistical tangent space. The tangent set consists of attainable scores; the tangent space also includes their linear combinations and $L^2(P_f)$ limits.

Let $g$ be a bounded function with $P_{f}g=0$. A bounded density tilt realizes this direction:These are nonnegative probability density functions, since $\int f_t=1+t{P}_{f}g=1$. The uniform Taylor expansion of the square root givesThe squared L2 norm of this remainder is $O(t^4{P}_f{g}^4)=o(t^2)$, since $g$ is bounded. Thus the statistical path is differentiable in quadratic mean with score function $g$.

Conversely every score function is centered by the mean-zero score identity under quadratic-mean differentiability. Choosing all these bounded density tilts therefore gives the statistical tangent setThis is a valid choice of statistical tangent set; it does not assert that every possible score function in the unrestricted density model is bounded.

Consider any differentiable-in-quadratic-mean path with score function $g$. Put ${\delta }_t=\sqrt{f_t}-\sqrt{f}=(t/2)g\sqrt{f}+r_t$, where $\Vert r_t{\Vert }_2=o(|t|)$. The quadratic-mean to L1 density derivative follows fromBy the Cauchy-Schwarz inequality,Since $a$ is a bounded function, multiplying this L1 norm bound by $\Vert a{\Vert }_{\infty }$ provesThe last equality uses $P_{f}g=0$. The derivative is a bounded linear functional of $g\in L^2(P_f)$, so the required pathwise differentiability of a statistical functional holds, in particular relative to the statistical tangent set from part (b). Its derivative isFor the explicit bounded density tilts in part (b), this derivative is also obtained by direct integration, with no remainder term.

For a pathwise differentiable statistical functional, an influence-function representer is a mean-zero function $\phi \in L^2(P)$ such that every admissible score function $g$ satisfies $D{\psi }_P(g)=P(\phi g)$. The efficient influence function, also called the canonical gradient, is the unique such representer in the statistical tangent space. Equivalently, it is the orthogonal projection of any representer onto that statistical tangent space. The Pythagorean theorem in an inner-product space shows that it has the smallest squared L2 norm among all representers.

Here the statistical tangent space is all of $L_0^2(P_f)$. To verify the closure explicitly, take $h\in L_0^2(P_f)$, truncate it to $h_m=\max (-m,\min (h,m))$, and set $g_m=h_m-P_f{h}_m$. Then $g_m$ is bounded and centered, and $g_m\to h$ in $L^2(P_f)$, by dominated convergence and the Cauchy-Schwarz inequality. Part (c) supplies a representer already in this space. HenceIts variance is $P_f{a}^2-(P_{f}a{)}^2$.

The closure must be taken in the density-weighted space $L^2(P_f)$. An unweighted reading of $L^2[0,1]$ in the printed hint is false. For example, when $f(u)=2u$, the function $h(u)=u^{-3/4}-8/5$ has $P_{f}h=0$ and $P_f{h}^2=36/25$, so bounded centered truncations converge to it in $L^2(P_f)$; nevertheless $h\notin L^2([0,1],du)$. This illustrates density of bounded centered scores and fixes the measure in the closure statement.

For the bounded density tilt $f_t=f(1+tg)$, differentiate the polynomial in $t$:Since every score function is centered, the centered representer isA bounded baseline $f$ makes this a bounded mean-zero function, hence an element of $L_0^2(P_f)$. Thus it is the efficient influence function for the density fourth-power functional relative to the regular bounded density tilts. Its squared L2 norm is $16\{\int f^7-(\int f^4{)}^2\}$.

A bounded baseline alone does not make this functional differentiable along every quadratic-mean differentiable path. The derivative above is the intended regular-path answer. To see the need for the qualification, let $f_0=1$ and put $b(s)=6s(1-s)$ on $[0,1]$, extended by zero outside. For $0<|t|<1$, defineEach $f_t$ is a nonnegative continuous probability density function, because its narrow bump has mass $t^4$. Moreover,It is therefore a differentiable-in-quadratic-mean path with zero score function. YetThe density fourth-power functional is not even continuous along this statistical path, although every $f_t$ is individually bounded. Consequently no efficient influence function represents derivatives over the unrestricted class of all such paths.

One sufficient additional condition is a common bound $f_t,f\le M$ for all small $t$. Taylor expansion then bounds the fourth-power remainder by $6M^2(f_t-f{)}^2$, whileTogether with the quadratic-mean to L1 density derivative, this gives $\psi (P_{f_t})-\psi (P_f)=t{P}_f({\phi }_{f}g)+o(|t|)$. Under this local bound, or when the chosen statistical paths are the bounded density tilts, the boxed canonical gradient is fully justified. The counterexample is a spike obstruction to density-power differentiability.

Write $e=y-g_{\theta }(x)$ and $h_{\theta }(x)=\partial g_{\theta }(x)/\partial \theta$. Holding the nuisance parameter $\eta$ fixed, the parametric score function is the derivative with respect to $\theta$ of the one-observation log-likelihood. The joint probability density function isBecause $\partial e/\partial \theta =-h_{\theta }(x)$, differentiating givesFor instance $E_{\eta }{h}_{\theta }(X{)}^2<\infty$ ensures a square-integrable score function. Its expected value is zero by independence and the centered normal distribution of the error. This is the Gaussian regression score.

Vary only the probability density function of $X$, using ${\eta }_t=\eta (1+tb)$ with bounded $b$ and $E_{\eta }b(X)=0$. The nuisance score function is $b(X)$, giving the statistical tangent setIts closed linear span is the nuisance tangent space of all centered $L^2(\eta )$ functions of $X$, by density of bounded centered scores.

The conditional expectation of the parametric score function given $X$ is zero:It is therefore orthogonal to this nuisance tangent space. Its orthogonal projection onto that space vanishes, so the efficient score is unchanged. By independence and $E{\epsilon }^2={\sigma }^2$,These equal the parametric score function and Fisher information when $\eta$ is known. There is no loss of information from the unknown covariate density. This is adaptivity to an unknown covariate distribution; it follows from score orthogonality, without having to estimate the nuisance parameter.