Unbiased estimator

An estimator $T$ of a parameter $\theta$ is unbiased when ${\mathbb{E}}_{\theta }T=\theta$ for every parameter value in the model.

For independent $X_i=\theta +Y_i$ with $Y_i$ exponentially distributed at known rate $\lambda$, the joint likelihood factors through the sample minimum $X_{(1)}$. Its excess over the endpoint is exponential with rate $n\lambda$, because $\mathbb{P}(\min Y_i>t)=e^{-n\lambda t}$. Thus the displayed estimator is an unbiased estimator, with variance $(n\lambda {)}^{-2}$. Even if $\theta \ge 0$, the estimator can be negative; truncation at zero introduces bias.

An unbiased estimator $T$ is uniformly minimum-variance unbiased if its variance is no larger than that of every other unbiased estimator of the same parameter, for every parameter value. The Lehmann–Scheffé theorem gives such an estimator whenever an unbiased function of a complete sufficient statistic exists. Attaining the Cramér-Rao bound is sufficient for this property under its hypotheses, but is not necessary: for $T\sim \Gamma (n,\lambda )$ with $n>2$, $(n-1)/T$ is unbiased and has variance ${\lambda }^2/(n-2)$, while the information bound is ${\lambda }^2/n$. Completeness follows from uniqueness of the Laplace transform of $h(t)t^{n-1}$, so the estimator is still uniformly optimal among unbiased estimators.

An estimator is conditionally unbiased for a parameter if its expectation, conditional on the specified observation or selection event, equals that parameter. Unconditional unbiasedness does not imply this property after selection. Independent new-stage data and the Rao-Blackwell theorem can provide useful constructions.

A UMVCUE has minimum conditional variance among all estimators unbiased in the selected experiment for every parameter. For independent Gaussian stage estimates $A,V$ with information $I_1,J$, continuation $A\ge c$, total information $I_2=I_1+J$ and pooled estimate $T$, Rao-Blackwellization of the fresh estimate gives $T-(I_{1}s/J)\varphi ((T-c)/s)/\Phi ((T-c)/s)$, where $s^2=1/I_1-1/I_2$. The conditional family has complete sufficient statistic $T$, so the Lehmann–Scheffé theorem establishes the optimum. A conditional-likelihood bias correction does not automatically have this unbiasedness property.

In a linear model $Y=X\beta +\epsilon$, a linear estimator has the form $AY$. It is unbiased for $C\beta$ exactly when $AX=C$. A best linear unbiased estimator minimizes its covariance, or its variance for a scalar target, among all estimators satisfying this constraint.

A best linear unbiased estimator minimizes covariance among linear estimators unbiased for the same target. In $Y=X\beta +\epsilon$, assume $\mathbb{E}\epsilon =0$, $X$ is a full-column-rank design matrix and the known covariance matrix $\Sigma$ is a positive-definite matrix. The best linear unbiased estimator of the coefficient vector is $(X^T{\Sigma }^{-1}X{)}^{-1}{X}^T{\Sigma }^{-1}Y$. A spatial trend estimate excludes the correlated residual prediction included in universal kriging.

For $X\sim N(\mu 1,C)$ with a known positive-definite matrix $C$, the maximum-likelihood estimator of $\mu$ is $\hat{\mu }=(1^T{C}^{-1}X)/(1^T{C}^{-1}1)$. It is unbiased with variance $(1^T{C}^{-1}1{)}^{-1}$, attaining the Cramér-Rao bound. It is also the best linear unbiased estimator. Pairwise admissible correlation coefficients alone do not ensure that $C$ is a valid covariance matrix; the entire matrix must be symmetric and a positive semidefinite matrix. Inverse-based formulas require positive definiteness.

For independent unbiased estimators $X_i$ of a common mean with known positive variances $v_i$, the linear unbiased estimator with smallest variance has weights $w_i=v_i^{-1}/{\sum }_j{v}_j^{-1}$ and variance $({\sum }_j{v}_j^{-1}{)}^{-1}$. Unbiasedness requires ${\sum }_i{w}_i=1$. Minimizing ${\sum }_i{v}_i{w}_i^2$ by a Lagrange multiplier gives the weights, and the strictly positive diagonal Hessian matrix proves a unique global minimum. This conclusion requires no normality assumption.

A weight $w_i=1/v_i$ gives more weight to a more precise estimate. In random-effects meta-analysis the variance of the corresponding marginal distribution is $v_i+{\tau }^2$, giving weight $(v_i+{\tau }^2{)}^{-1}$. A percentage weight divides the individual weight by the sum of all weights.