Past exam of the mathematics course of the University of Cambridge / 2012 / iii / Paper 36 / 2

Choose a collection of statistical paths through $P_f$ that are differentiable in quadratic mean. A statistical tangent set at $P_f$ is the set of their score functions. In particular each member belongs to $L_0^2(P_f)$, by the mean-zero score identity under quadratic-mean differentiability.

A statistical tangent set records which first-order directions the chosen statistical paths can realize. Its closed linear span in L2 space is the statistical tangent space. The tangent set consists of attainable scores; the tangent space also includes their linear combinations and $L^2(P_f)$ limits.

Let $g$ be a bounded function with $P_{f}g=0$. A bounded density tilt realizes this direction:These are nonnegative probability density functions, since $\int f_t=1+t{P}_{f}g=1$. The uniform Taylor expansion of the square root givesThe squared L2 norm of this remainder is $O(t^4{P}_f{g}^4)=o(t^2)$, since $g$ is bounded. Thus the statistical path is differentiable in quadratic mean with score function $g$.

Conversely every score function is centered by the mean-zero score identity under quadratic-mean differentiability. Choosing all these bounded density tilts therefore gives the statistical tangent setThis is a valid choice of statistical tangent set; it does not assert that every possible score function in the unrestricted density model is bounded.

Consider any differentiable-in-quadratic-mean path with score function $g$. Put ${\delta }_t=\sqrt{f_t}-\sqrt{f}=(t/2)g\sqrt{f}+r_t$, where $\Vert r_t{\Vert }_2=o(|t|)$. The quadratic-mean to L1 density derivative follows fromBy the Cauchy-Schwarz inequality,Since $a$ is a bounded function, multiplying this L1 norm bound by $\Vert a{\Vert }_{\infty }$ provesThe last equality uses $P_{f}g=0$. The derivative is a bounded linear functional of $g\in L^2(P_f)$, so the required pathwise differentiability of a statistical functional holds, in particular relative to the statistical tangent set from part (b). Its derivative isFor the explicit bounded density tilts in part (b), this derivative is also obtained by direct integration, with no remainder term.

For a pathwise differentiable statistical functional, an influence-function representer is a mean-zero function $\phi \in L^2(P)$ such that every admissible score function $g$ satisfies $D{\psi }_P(g)=P(\phi g)$. The efficient influence function, also called the canonical gradient, is the unique such representer in the statistical tangent space. Equivalently, it is the orthogonal projection of any representer onto that statistical tangent space. The Pythagorean theorem in an inner-product space shows that it has the smallest squared L2 norm among all representers.

Here the statistical tangent space is all of $L_0^2(P_f)$. To verify the closure explicitly, take $h\in L_0^2(P_f)$, truncate it to $h_m=\max (-m,\min (h,m))$, and set $g_m=h_m-P_f{h}_m$. Then $g_m$ is bounded and centered, and $g_m\to h$ in $L^2(P_f)$, by dominated convergence and the Cauchy-Schwarz inequality. Part (c) supplies a representer already in this space. HenceIts variance is $P_f{a}^2-(P_{f}a{)}^2$.

The closure must be taken in the density-weighted space $L^2(P_f)$. An unweighted reading of $L^2[0,1]$ in the printed hint is false. For example, when $f(u)=2u$, the function $h(u)=u^{-3/4}-8/5$ has $P_{f}h=0$ and $P_f{h}^2=36/25$, so bounded centered truncations converge to it in $L^2(P_f)$; nevertheless $h\notin L^2([0,1],du)$. This illustrates density of bounded centered scores and fixes the measure in the closure statement.

For the bounded density tilt $f_t=f(1+tg)$, differentiate the polynomial in $t$:Since every score function is centered, the centered representer isA bounded baseline $f$ makes this a bounded mean-zero function, hence an element of $L_0^2(P_f)$. Thus it is the efficient influence function for the density fourth-power functional relative to the regular bounded density tilts. Its squared L2 norm is $16\{\int f^7-(\int f^4{)}^2\}$.

A bounded baseline alone does not make this functional differentiable along every quadratic-mean differentiable path. The derivative above is the intended regular-path answer. To see the need for the qualification, let $f_0=1$ and put $b(s)=6s(1-s)$ on $[0,1]$, extended by zero outside. For $0<|t|<1$, defineEach $f_t$ is a nonnegative continuous probability density function, because its narrow bump has mass $t^4$. Moreover,It is therefore a differentiable-in-quadratic-mean path with zero score function. YetThe density fourth-power functional is not even continuous along this statistical path, although every $f_t$ is individually bounded. Consequently no efficient influence function represents derivatives over the unrestricted class of all such paths.

One sufficient additional condition is a common bound $f_t,f\le M$ for all small $t$. Taylor expansion then bounds the fourth-power remainder by $6M^2(f_t-f{)}^2$, whileTogether with the quadratic-mean to L1 density derivative, this gives $\psi (P_{f_t})-\psi (P_f)=t{P}_f({\phi }_{f}g)+o(|t|)$. Under this local bound, or when the chosen statistical paths are the bounded density tilts, the boxed canonical gradient is fully justified. The counterexample is a spike obstruction to density-power differentiability.