Estimator ($\hat{\theta }$)

An estimator is a function of observed data used to estimate a parameter or a specified function of parameters. Its definition cannot involve the unknown parameter. Its bias, variance, and expected loss describe its performance under repeated sampling.

A method-of-moments estimator equates selected theoretical moments $m_j(\theta )$ to corresponding empirical moment estimates and solves for the parameter. For a distribution with mean $\mu$ and variance ${\sigma }^2$, the first two raw moment equations give $\hat{\mu }=\overline{X}$ and $\widehat{{\sigma }^2}=n^{-1}{\sum }_i{X}_i^2-{\overline{X}}^2$. Existence, uniqueness and consistency require appropriate identifiability and moment assumptions; unbiased empirical moments do not make nonlinear parameter estimates unbiased. This statistical estimation procedure differs from the method of moments in probability for proving convergence in distribution.

A U-statistic averages a symmetric function of $m$ distinct sample observations over all unordered $m$-tuples. For independent and identically distributed observations it is an unbiased estimator of $\mathbb{E}H(X_1,\dots ,X_m)$. Omitting repeated indices can remove diagonal contributions in quadratic plug-in estimators.

For a symmetric degree-$m$ U-statistic kernel, the population functional is $T(F)=\int hd{F}^m$. Contaminating each of the $m$ identical distribution arguments and differentiating gives the displayed influence function. It equals $m$ times the first Hoeffding projection and gives leading asymptotic variance $m^2\mathrm{Var}(h_1(X))/n$. If that projection vanishes, the usual root-$n$ normal limit degenerates and higher-order projections determine the limiting law.

A U-statistic kernel is an integrable function of a fixed number r of observations whose expectation is the parameter functional to be estimated. Exchangeability permits symmetrization without changing that expectation. This is a function of r sample values, rather than the one-displacement kernel for density estimation.

The symmetric kernel $h(x,y)=(x-y{)}^2/2$ has expectation equal to the population variance for independent identically distributed observations with finite second moment. Its U-statistic reduces to the displayed unbiased sample variance, using ${\sum }_{i<j}(X_i-X_j{)}^2=n{\sum }_i(X_i-\overline{X}{)}^2$.

Exchangeability of an independent identically distributed sample makes every permuted kernel have the same expectation. Averaging over permutations therefore leaves the target functional unchanged and gives a symmetric kernel. The U-statistic then averages this kernel over the unordered r-element sample subsets.

For an iid sample and symmetric square-integrable kernel $h$ of fixed order $m$, put $\eta =\mathbb{E}h(X_1,\dots ,X_m)$ and $h_1(x)=\mathbb{E}[h(x,X_2,\dots ,X_m)]-\eta$. If ${\zeta }_1=\mathrm{Var}(h_1(X_1))>0$, the displayed limit holds. The first Hoeffding projection is $(m/n){\sum }_i{h}_1(X_i)$; its orthogonal remainder is $O_{L^2}(n^{-1})$, giving the limit by the ordinary central limit theorem. Degenerate kernels require a different limiting theory.

The displayed sum is the first-order projection of a centred U-statistic onto sums of functions of individual observations. Its summands are independent and have mean zero. Subtracting it leaves a remainder with conditional expectation zero in each individual observation, a useful decomposition for variance and normal-limit calculations.