= Uncertainty in estimated inverse-variance weights
<Uncertainty in estimated inverse-variance weights> arises when the <variances> used in <inverse-variance weighting> are themselves estimated. A very small estimated <variance> can give one observation or group a large random weight. The familiar <variance> formula for fixed, known weights cannot simply be reused with estimated <variances> treated as exact. NIST specifically warns that weights estimated from few replicates can substantially and unpredictably affect a <weighted least squares> analysis.<Weighted Least Squares by NIST>[e-Handbook §4.1.4.3, Disadvantages of Weighted Least Squares]{ref}
Suppose independent groups contain $n_i$ independent <normal random variables> with common mean $\theta$ and group <variance> $\sigma_i^2$. Let $\overline X_i$ and $S_i^2$ be the group <sample mean> and unbiased <sample variance>, and form
$$
\widehat w_i=\frac{n_i/S_i^2}{\sum_j n_j/S_j^2},\qquad
\widehat\theta=\sum_i\widehat w_i\overline X_i.
$$
For normal samples, each <sample mean> is independent of its <sample variance>. Conditional on the estimated weights, the estimator remains unbiased and has
$$
\operatorname{Var}(\widehat\theta\mid\widehat w)
=\sum_i\widehat w_i^2\frac{\sigma_i^2}{n_i}.
$$
Thus the <law of total variance> gives unconditional <variance> equal to the expectation of this expression. The true $\sigma_i^2$ still appear; replacing them by the same random $S_i^2$ does not establish the actual conditional <variance>. Outside the independence assumptions, even conditional unbiasedness can fail.
An original two-group example takes $n_1=n_2=5$ and true $\sigma_1^2=\sigma_2^2=4$, so each group mean has <variance> $0.8$. If observed sample <variances> are $0.25$ and $4$, the estimated normalized weights are $16/17$ and $1/17$. For group means $10$ and $14$, the combined estimate is $174/17\approx10.2353$. The plug-in <variance> is
$$
\widehat V=\frac1{5/0.25+5/4}=\frac4{85}\approx0.04706,
$$
but the true conditional <variance> under the stipulated model is
$$
V=0.8\left[\left(\frac{16}{17}\right)^2+\left(\frac1{17}\right)^2\right]
\approx0.71142.
$$
The corresponding standard deviations are about $0.2169$ and $0.8435$. Equal weights would have <variance> $0.4$. An accidentally small estimated <variance> has both overstated precision and moved the combination away from the optimal equal weights in this example.
<Reciprocal moments of an estimated variance> explain the unstable precision estimate. For a normal sample with $\nu=n-1$, $Z=\nu S^2/\sigma^2$ has a <chi-squared distribution>. Integrating its density gives
$$
\mathbb E[Z^{-r}]=2^{-r}\frac{\Gamma(\nu/2-r)}{\Gamma(\nu/2)},\qquad \nu>2r.
$$
Consequently $\mathbb E[1/S^2]=\nu/[(\nu-2)\sigma^2]$ for $\nu>2$, while the second moment requires $\nu>4$. The density and its gamma-distribution parameterization are documented in the primary SciPy implementation notes.<SciPy 1.16.2 documentation>[scipy.stats.chi2, Notes, density and gamma parameterization]{ref} With $n=5$, the inverse <sample variance> has a finite mean twice the true precision and an infinite <variance>; the normalized weights remain bounded, so this does not imply their <variance> is infinite.
More replication or a justified smooth <variance> model can stabilize weights. NIST describes direct replicate-based weight estimation and variance-function modeling as distinct approaches.<Weighted Least Squares by NIST>[e-Handbook §4.4.5.2, Direct Estimation of Weights and Estimating Weights with a Variance Function]{ref} A proposed validation repeats the complete weight-estimation and combination procedure on simulated or replicated groups, checking actual interval coverage as well as point estimates. A <bootstrap> must reproduce the weight-estimation stage; holding the fitted weights fixed tests a different procedure.
Back to article page
