Unbiased estimator

An estimator $T$ of a parameter $\theta$ is unbiased when ${\mathbb{E}}_{\theta }T=\theta$ for every parameter value in the model.

An estimator is conditionally unbiased for a parameter if its expectation, conditional on the specified observation or selection event, equals that parameter. Unconditional unbiasedness does not imply this property after selection. Independent new-stage data and the Rao-Blackwell theorem can provide useful constructions.

A UMVCUE has minimum conditional variance among all estimators unbiased in the selected experiment for every parameter. For independent Gaussian stage estimates $A,V$ with information $I_1,J$, continuation $A\ge c$, total information $I_2=I_1+J$ and pooled estimate $T$, Rao-Blackwellization of the fresh estimate gives $T-(I_{1}s/J)\varphi ((T-c)/s)/\Phi ((T-c)/s)$, where $s^2=1/I_1-1/I_2$. The conditional family has complete sufficient statistic $T$, so the Lehmann–Scheffé theorem establishes the optimum. A conditional-likelihood bias correction does not automatically have this unbiasedness property.

In a linear model $Y=X\beta +\epsilon$, a linear estimator has the form $AY$. It is unbiased for $C\beta$ exactly when $AX=C$. A best linear unbiased estimator minimizes its covariance, or its variance for a scalar target, among all estimators satisfying this constraint.

A best linear unbiased estimator minimizes covariance among linear estimators unbiased for the same target. In $Y=X\beta +\epsilon$, assume $\mathbb{E}\epsilon =0$, $X$ is a full-column-rank design matrix and the known covariance matrix $\Sigma$ is a positive-definite matrix. The best linear unbiased estimator of the coefficient vector is $(X^T{\Sigma }^{-1}X{)}^{-1}{X}^T{\Sigma }^{-1}Y$. A spatial trend estimate excludes the correlated residual prediction included in universal kriging.

For $X\sim N(\mu 1,C)$ with a known positive-definite matrix $C$, the maximum-likelihood estimator of $\mu$ is $\hat{\mu }=(1^T{C}^{-1}X)/(1^T{C}^{-1}1)$. It is unbiased with variance $(1^T{C}^{-1}1{)}^{-1}$, attaining the Cramer-Rao bound. It is also the best linear unbiased estimator. Pairwise admissible correlation coefficients alone do not ensure that $C$ is a valid covariance matrix; the entire matrix must be symmetric and a positive semidefinite matrix. Inverse-based formulas require positive definiteness.

For independent unbiased estimators $X_i$ of a common mean with known positive variances $v_i$, the linear unbiased estimator with smallest variance has weights $w_i=v_i^{-1}/{\sum }_j{v}_j^{-1}$ and variance $({\sum }_j{v}_j^{-1}{)}^{-1}$. Unbiasedness requires ${\sum }_i{w}_i=1$. Minimizing ${\sum }_i{v}_i{w}_i^2$ by a Lagrange multiplier gives the weights, and the strictly positive diagonal Hessian matrix proves a unique global minimum. This conclusion requires no normality assumption.

A weight $w_i=1/v_i$ gives more weight to a more precise estimate. In random-effects meta-analysis the variance of the corresponding marginal distribution is $v_i+{\tau }^2$, giving weight $(v_i+{\tau }^2{)}^{-1}$. A percentage weight divides the individual weight by the sum of all weights.