Semiparametric statistics

Semiparametric statistics studies a statistical model with a finite-dimensional target statistical parameter and an infinite-dimensional nuisance parameter. For example, the partially linear model $Y=\beta A+g(X)+\epsilon$, with $\mathbb{E}[\epsilon \mid A,X]=0$, targets the scalar $\beta$ while leaving the function $g$ and the covariate distribution unrestricted.

Writing $U=A-\mathbb{E}[A\mid X]$ and $V=Y-\mathbb{E}[Y\mid X]$ removes $g$: $V=\beta U+\epsilon$. If the needed second moments exist and $\mathbb{E}{U}^2>0$, then $\beta =\mathbb{E}[UV]/\mathbb{E}{U}^2$. The positivity condition expresses the residual variation needed to identify the target.

A nuisance tangent space describes first-order changes the nuisance can imitate. Projecting a parametric score function away from that space gives the efficient score and efficient information. Thus a finite-dimensional target can sometimes be estimated efficiently without fully specifying an infinite-dimensional distribution; the allowed model and its regularity determine whether this is possible.

If the parametric score function has zero conditional expectation given a covariate $X$, it is orthogonal to the centered functions of $X$ that form the covariate-density nuisance tangent space. Its efficient score then equals its parametric score function, and the unknown covariate distribution causes no loss of Fisher information. This applies to Gaussian regression scores and more generally to regular conditional models with unrestricted covariate distribution and no additional nuisance components.

The nuisance tangent space is the closed linear span of score functions from statistical paths that vary the nuisance parameter while fixing the target statistical parameter. It is a closed subspace of a Hilbert space inside $L_0^2(P)$. Removing its component from a parametric score function leaves the information that nuisance variation cannot imitate.

For independent-error regression with a zero-mean error, bounded density paths $f_t=f(1+t\gamma )$ must preserve both normalization and the first moment. Their score functions therefore satisfy two constraints. When $0<E_f{\epsilon }^2<\infty$, truncation followed by two small bounded moment corrections shows these scores are dense in the displayed closed subspace of a Hilbert space. By independence, they are orthogonal to every $\epsilon \psi (X)$ with $\psi \in L^2(v)$. The same constraints apply to other paths only under regularity permitting differentiation of the first moment.

The efficient score is the residual after orthogonal projection of the parametric score function onto the nuisance tangent space. It is centered and orthogonal to every nuisance direction. Its squared L2 norm is the efficient information.

With unknown independent covariate and centered error distributions, take the regular mean-preserving error tangent space, let $h={\partial }_{\theta }{g}_{\theta }$, $\rho =-f^{\prime }/f$ and ${\tau }^2=E_f{\epsilon }^2>0$. Under $E_f\rho =0$, $E_f(\epsilon \rho )=1$ and finite second moments, orthogonal projection removes $(E_{v}h)(\rho -\epsilon /{\tau }^2)$ from $h\rho$. Thus the efficient score and efficient information are $\tilde{\ell }=(h-E_{v}h)\rho +(E_{v}h)\epsilon /{\tau }^2$ and $\tilde{I}={\mathrm{Var}}_v(h)E_f{\rho }^2+(E_{v}h{)}^2/{\tau }^2$. The reduction to $h\epsilon /{\tau }^2$ is valid for a normal distribution of errors or for constant $h$, but need not hold otherwise. Centered covariates and logistic distribution errors give the counterexample $\tilde{\ell }=X\tanh (\epsilon /2)$.

The decomposition $\dot{\ell }={\Pi }_{\mathcal{N}}\dot{\ell }+\tilde{\ell }$ is orthogonal. Taking the inner product with the efficient score gives $P(\dot{\ell }\tilde{\ell })=\tilde{I}$. Centering of the efficient score follows because the nuisance tangent space and the parametric score function lie in the closed mean-zero L2 space.

The efficient information for a scalar target statistical parameter is the squared L2 norm of its efficient score. It cannot exceed the parametric Fisher information, by the Pythagorean theorem in an inner-product space. It can vanish when first-order target variation can be reproduced by nuisance variation.

A semiparametric estimator targets a finite-dimensional parameter while estimating or otherwise accommodating an infinite-dimensional nuisance component.

A partially linear model combines a linear coefficient with an unrestricted nuisance function, for example $\mathbb{E}[Y\mid A,X]=\beta A+g(X)$.