 Uncertainty in estimated inverse-variance weights

Uncertainty in estimated inverse-variance weights arises when the variances used in inverse-variance weighting are themselves estimated. A very small estimated variance can give one observation or group a large random weight. The familiar variance formula for fixed, known weights cannot simply be reused with estimated variances treated as exact. NIST specifically warns that weights estimated from few replicates can substantially and unpredictably affect a weighted least squares analysis.^[ref]

Suppose independent groups contain $n_i$ independent normal random variables with common mean $\theta$ and group variance ${\sigma }_i^2$. Let ${\overline{X}}_i$ and $S_i^2$ be the group sample mean and unbiased sample variance, and formFor normal samples, each sample mean is independent of its sample variance. Conditional on the estimated weights, the estimator remains unbiased and hasThus the law of total variance gives unconditional variance equal to the expectation of this expression. The true ${\sigma }_i^2$ still appear; replacing them by the same random $S_i^2$ does not establish the actual conditional variance. Outside the independence assumptions, even conditional unbiasedness can fail.

An original two-group example takes $n_1=n_2=5$ and true ${\sigma }_1^2={\sigma }_2^2=4$, so each group mean has variance $0.8$. If observed sample variances are $0.25$ and $4$, the estimated normalized weights are $16/17$ and $1/17$. For group means $10$ and $14$, the combined estimate is $174/17\approx 10.2353$. The plug-in variance isbut the true conditional variance under the stipulated model isThe corresponding standard deviations are about $0.2169$ and $0.8435$. Equal weights would have variance $0.4$. An accidentally small estimated variance has both overstated precision and moved the combination away from the optimal equal weights in this example.

Reciprocal moments of an estimated variance explain the unstable precision estimate. For a normal sample with $\nu =n-1$, $Z=\nu S^2/{\sigma }^2$ has a chi-squared distribution. Integrating its density givesConsequently $\mathbb{E}[1/S^2]=\nu /[(\nu -2){\sigma }^2]$ for $\nu >2$, while the second moment requires $\nu >4$. The density and its gamma-distribution parameterization are documented in the primary SciPy implementation notes.^[ref] With $n=5$, the inverse sample variance has a finite mean twice the true precision and an infinite variance; the normalized weights remain bounded, so this does not imply their variance is infinite.

More replication or a justified smooth variance model can stabilize weights. NIST describes direct replicate-based weight estimation and variance-function modeling as distinct approaches.^[ref] A proposed validation repeats the complete weight-estimation and combination procedure on simulated or replicated groups, checking actual interval coverage as well as point estimates. A bootstrap must reproduce the weight-estimation stage; holding the fitted weights fixed tests a different procedure.