Statistical tangent set (${\dot{\mathcal{P}}}_P$)

A statistical tangent set is the collection of score functions attained by a specified family of differentiable-in-quadratic-mean paths through $P$. The choice of paths is part of the definition. Its elements lie in the mean-zero L2 space, but the set need not already be a closed vector subspace.

For bounded measurable $g$ with $Pg=0$, the formula $p_t=p(1+tg)$ defines a normalized positive probability density function for $|t|\Vert g{\Vert }_{\infty }<1$. A uniform Taylor expansion of the square root proves differentiability in quadratic mean with score function $g$: the squared remainder is $O(t^{4}P{g}^4)$. Such paths realize all bounded centered directions of an unrestricted density model.

For $h\in L_0^2(P)$, truncate $h$ to $h_m=\max (-m,\min (h,m))$ and subtract $P{h}_m$. Dominated convergence gives $h_m\to h$ in L2 space, and the Cauchy-Schwarz inequality gives $P{h}_m\to 0$. Thus bounded centered directions are dense in the mean-zero L2 space. The measure defining the L2 norm is essential: density-weighted L2 space can contain functions outside the unweighted Lebesgue space.

The statistical tangent space is the closed linear span in L2 space of a statistical tangent set. Taking this closure makes orthogonal projection available and ensures that a continuous derivative specified on attainable score functions extends to the whole space.