Past exam of the mathematics course of the University of Cambridge / 2012 / iii / Paper 36 / 1

A statistical path through $P$ is a family $t\mapsto P_t\in \mathcal{P}$, defined on an open interval containing zero, with $P_0=P$. Write $p_t=d{P}_t/d\mu$ and $p=p_0$. Its differentiability in quadratic mean at zero means that there is a square-integrable function $g\in L^2(P)$ such thatThe function $g$ is the score function of the statistical path. Thus it is the derivative of the square-root probability density function, multiplied by two and divided by $\sqrt{p}$ where $p>0$. The score function is determined only $P$-almost everywhere. The displayed definition also controls any probability mass entering a region where $p=0$; a pointwise derivative of the log probability density function alone would not do that. The required derivative is in the square-root-density $L^2$ norm.

Put $r_t=\sqrt{p_t}-\sqrt{p}-(t/2)g\sqrt{p}$. By differentiability in quadratic mean, $\Vert r_t{\Vert }_{L^2(\mu )}=o(|t|)$ andin L2 space. The second limit follows because the first gives $\Vert \sqrt{p_t}-\sqrt{p}{\Vert }_2=O(|t|)$.

Both probability density functions integrate to one. Consequently the L2 inner product satisfiesThe Cauchy-Schwarz inequality permits passage to the limit in this inner product, givingThus every score function is a mean-zero function. This proof uses square-root differentiability in quadratic mean, without requiring differentiation under the density integral.

Fix $P=P_{\theta ,\eta }$. Let $\mathcal{N}$ be the nuisance tangent space: the closed linear span in $L^2(P)$ of score functions of statistical paths that vary only the nuisance parameter $\eta$. By the preceding argument, $\mathcal{N}$ is contained in the mean-zero L2 space $L_0^2(P)$.

Let ${\Pi }_{\mathcal{N}}$ denote orthogonal projection onto this closed subspace of a Hilbert space. The efficient score and scalar efficient information areThe efficient score is the component of the parametric score function that cannot be reproduced by changing the nuisance parameter. The efficient information is its squared L2 norm; it can be zero, so positivity must not be assumed in the definition.

The map $h\mapsto Ph$ is a bounded linear functional on L2 space, since $|Ph|\le \Vert h{\Vert }_2$ by the Cauchy-Schwarz inequality. Its kernel, the mean-zero L2 space, is therefore closed. Every nuisance score function and the parametric score function $\dot{\ell }$ are centered, so both ${\Pi }_{\mathcal{N}}\dot{\ell }$ and $\tilde{\ell }$ are centered. HenceMoreover, the efficient score belongs to the orthogonal complement of the nuisance tangent space. Writing $\dot{\ell }=\tilde{\ell }+{\Pi }_{\mathcal{N}}\dot{\ell }$ givesThis is the efficient-score projection identity; it remains valid when the efficient information is zero.