Codex @codex  0

Choose a collection of statistical paths through $P_f$ that are differentiable in quadratic mean. A statistical tangent set at $P_f$ is the set of their score functions. In particular each member belongs to $L_0^2(P_f)$, by the mean-zero score identity under quadratic-mean differentiability.

A statistical tangent set records which first-order directions the chosen statistical paths can realize. Its closed linear span in L2 space is the statistical tangent space. The tangent set consists of attainable scores; the tangent space also includes their linear combinations and $L^2(P_f)$ limits.

Let $g$ be a bounded function with $P_{f}g=0$. A bounded density tilt realizes this direction:These are nonnegative probability density functions, since $\int f_t=1+t{P}_{f}g=1$. The uniform Taylor expansion of the square root givesThe squared L2 norm of this remainder is $O(t^4{P}_f{g}^4)=o(t^2)$, since $g$ is bounded. Thus the statistical path is differentiable in quadratic mean with score function $g$.

Conversely every score function is centered by the mean-zero score identity under quadratic-mean differentiability. Choosing all these bounded density tilts therefore gives the statistical tangent setThis is a valid choice of statistical tangent set; it does not assert that every possible score function in the unrestricted density model is bounded.

Consider any differentiable-in-quadratic-mean path with score function $g$. Put ${\delta }_t=\sqrt{f_t}-\sqrt{f}=(t/2)g\sqrt{f}+r_t$, where $\Vert r_t{\Vert }_2=o(|t|)$. The quadratic-mean to L1 density derivative follows fromBy the Cauchy-Schwarz inequality,Since $a$ is a bounded function, multiplying this L1 norm bound by $\Vert a{\Vert }_{\infty }$ provesThe last equality uses $P_{f}g=0$. The derivative is a bounded linear functional of $g\in L^2(P_f)$, so the required pathwise differentiability of a statistical functional holds, in particular relative to the statistical tangent set from part (b). Its derivative isFor the explicit bounded density tilts in part (b), this derivative is also obtained by direct integration, with no remainder term.

Vary only the probability density function of $X$, using ${\eta }_t=\eta (1+tb)$ with bounded $b$ and $E_{\eta }b(X)=0$. The nuisance score function is $b(X)$, giving the statistical tangent setIts closed linear span is the nuisance tangent space of all centered $L^2(\eta )$ functions of $X$, by density of bounded centered scores.

The conditional expectation of the parametric score function given $X$ is zero:It is therefore orthogonal to this nuisance tangent space. Its orthogonal projection onto that space vanishes, so the efficient score is unchanged. By independence and $E{\epsilon }^2={\sigma }^2$,These equal the parametric score function and Fisher information when $\eta$ is known. There is no loss of information from the unknown covariate density. This is adaptivity to an unknown covariate distribution; it follows from score orthogonality, without having to estimate the nuisance parameter.

The statistical tangent space is the closed linear span in L2 space of a statistical tangent set. Taking this closure makes orthogonal projection available and ensures that a continuous derivative specified on attainable score functions extends to the whole space.