Conservative variance estimator (${\mathbb{E}}_p\hat{V}\ge V_p$)

A conservative variance estimator, in the finite-sample expectation sense used here, is an estimator $\hat{V}$ of a sampling variance $V_{\theta }$ satisfyingfor every statistical parameter value $\theta$ in the stated statistical model. Assume $V_{\theta }$ and ${\mathbb{E}}_{\theta }|\hat{V}|$ are finite. In design-based inference, the same definition applies for every permitted fixed population, with the sampling design supplying the randomness. The bias of an estimator is then nonnegative for this target. Aronow and Samii use this expectation criterion for their correction to the Horvitz-Thompson variance estimator when some pairs cannot be sampled together.^[ref]

A samplewise variance upper bound would require the stronger inequality $\hat{V}\ge V_{\theta }$ almost surely. An expectation bound allows some estimates below $V_{\theta }$. It does not even require that every realized value be nonnegative; in practice a variance estimate is commonly chosen to be a nonnegative random variable.

One construction starts with an integrable random variable $W$ that is an unbiased estimator of $V_{\theta }$ and might take negative values. Its positive part $\hat{V}=W_{+}=\max (W,0)$ satisfies $W_{+}\ge W$, hence is a conservative variance estimator. The amount of upward bias of an estimator is exactlyThe identity follows from the positive part and negative part decomposition $W=W_{+}-(-W{)}_{+}$. The Horvitz-Thompson variance estimator supplies a concrete setting in which the untruncated unbiased estimator can be negative.

A deliberately constructed example proves that conservative variance estimation does not guarantee Wald interval coverage. Observe $Y$ from a normal distribution $N(\theta ,1)$ and an independent $B$ from a Bernoulli distribution with success probability $0<\epsilon <1$. Use $\hat{\theta }=Y$ and $\hat{V}=B/\epsilon$. This nonnegative random variable hasThus it is even an unbiased estimator of the sampling variance. Yet the usual Wald confidence intervalhas coverage probabilitywhere $\Phi$ is the standard normal distribution function. When $B=0$, the interval is the singleton $\{Y\}$, which contains $\theta$ with probability zero because $Y$ is a continuous random variable. When $B=1$, its conditional coverage probability is the bracketed expression. At $\epsilon =0.01$, overall coverage probability is at most 1 percent, despite the nominal 95 percent rule and the exactly normal centre.

This example concerns the logical strength of the expectation criterion. Controlling the distribution of an estimated standard error, for example through an appropriate consistent estimator in an asymptotic argument, is additional work needed to justify confidence interval coverage.

A conservative variance estimator bounds an expectation, whereas Wald confidence interval coverage is a probability of a joint event involving the centre and its random width. These are different requirements. In the cited sampling-design result, conservativeness means $E_{\theta }\hat{V}\ge {\mathrm{Var}}_{\theta }(\hat{\theta })$; the proposition is not a normal-distribution theorem.^[ref] The rare-large-variance counterexample in conservative variance estimator already shows how little that expectation alone controls coverage.

A separate original example isolates the distributional issue even when the variance is known exactly. Let $Y=\theta +X$, where $X$ is zero with probability $0.90$ and is each of $\sqrt{10}$ and $-\sqrt{10}$ with probability $0.05$. Symmetry gives $E_{\theta }Y=\theta$, while ${\mathrm{Var}}_{\theta }(Y)=0.1(10)=1$. Use the deterministic estimate $\hat{V}=1$: it is even a samplewise variance upper bound. Nevertheless,because $\sqrt{10}>1.96$. Accurate variance alone does not supply the normal distribution of a standardized error. NIST likewise distinguishes multiplying a standard uncertainty by a coverage factor from establishing a confidence level without distributional information.^[ref]

The tail assumptions needed for conservative Wald coverage may instead come from an asymptotic normal approximation with a consistent standard-error estimate, or from a justified finite-sample model. If only unbiasedness and a samplewise variance upper bound are available, Chebyshev's inequality gives a different, usually wider, confidence interval. One must establish the centre's bias, the variance condition, and the critical-value argument together.

The tail assumptions needed for conservative Wald coverage concern the distribution of the standardized estimation error and the realized scale of its interval. An expected-value bound on a conservative variance estimator does not establish either property. The discrete and rare-large-variance counterexamples in conservative variance estimation does not guarantee Wald interval coverage isolate those failures. NIST similarly makes a normal-model qualification before interpreting an expanded uncertainty as approximately 95 percent coverage.^[ref]

A precise sufficient asymptotic argument is the following. Let $V_n={\mathrm{Var}}_{\theta }({\hat{\theta }}_n)>0$, take ${\hat{V}}_n\ge 0$, and supposeThe first condition includes centring at the true parameter; asymptotically material bias would change it. The second condition is convergence in probability of a variance ratio, substantially more than an expectation inequality. By Slutsky's theorem, the Wald confidence interval with normal critical value $z={\Phi }^{-1}(1-\alpha /2)$ then has limiting coverage probabilityThe limiting normal distribution has no mass on the interval boundary, which justifies passing to that probability. This is asymptotic conservative Wald coverage under variance-ratio convergence, not a finite-sample guarantee.

For an original exactly normal comparison, let a sample mean have true variance $V_n$ and use the deterministic scale ${\hat{V}}_n=1.44V_n$. A nominal 95 percent Wald confidence interval uses a radius $1.2$ times the usual one and has exact coverage probability $2\Phi (1.2\times 1.959964)-1\approx 0.9813$. The calculation uses an actual normal sampling law and a fixed ratio; it cannot be inferred from $E{\hat{V}}_n\ge V_n$ alone. The normal cdf and inverse-cdf implementations provide the numerical evaluation.^[ref]

A samplewise variance upper bound supplies a separate finite-sample route when the centre is exactly normal, because each reported interval contains the known-variance normal interval. Without that shape assumption, use a justified tail inequality as in confidence bounds from a samplewise variance bound.

Asymptotic conservative Wald coverage under variance-ratio convergence requires the realized interval scale to be large enough in probability, together with a correctly centred limiting error law. The constant-ratio result in tail assumptions needed for conservative Wald coverage follows from Slutsky's theorem. A useful stronger formulation of the coverage conclusion needs no independence between centre and scale.

Let $Z_n=({\hat{\theta }}_n-\theta )/\sqrt{V_n}$ converge in distribution to a standard normal random variable, take $R_n={\hat{V}}_n/V_n\ge 0$, and let $z={\Phi }^{-1}(1-\alpha /2)>0$. If $R_n$ converges in probability to a constant $c\ge 1$, then for each $0<ϵ<1$, $\mathrm{Pr}(R_n<1-ϵ)\to 0$. ThereforeTaking the lower limit and then $ϵ\downarrow 0$ proves limiting coverage probability at least $1-\alpha$. In fact this lower-tail condition on $R_n$ suffices without convergence to one fixed constant. An expected-value inequality does not supply it; coverage failure under random variance-ratio limits can persist when the scale retains a substantial chance of being too small.

For an original comparison, choose $c=1.21$. Under an exactly centred normal limit, a nominal 95 percent Wald confidence interval has limiting coverage $2\Phi (1.1z)-1\approx 0.96891$. But if the standardized estimator instead converges to $N(0.8,1)$, the same scale givesA ten-percent increase in standard-error radius does not automatically cover an asymptotically material bias. These values use the normal cdf/ppf definitions.^[ref]

The result is pointwise in the assumed parameter/model sequence; it does not itself prove uniform coverage over a family of models or finite-sample accuracy. NIST's uncertainty guidance likewise conditions a confidence interpretation on the distributional justification, rather than on the multiplier alone.^[ref]

Coverage failure under random variance-ratio limits can persist even when the centre is exactly normal, the variance estimate is conservative in expectation, and its absolute error tends to zero. What fails is relative-scale concentration. Let $Z\sim N(0,1)$ and a positive random scale $R$ be independent, and defineThe variance ratio is $R$ at every sample size. Its nondegenerate distribution need not approach the constant required in asymptotic conservative Wald coverage under variance-ratio convergence.

Conditional on $R$, a normal critical value $z>0$ gives interval coverage $f(R)=2\Phi (z\sqrt{R})-1$. Therefore unconditional coverage is $Ef(R)$ under the independence assumption. A useful shape calculation isThe function is strictly concave. If $ER=1$, Jensen's inequality puts $Ef(R)\le f(1)$, with strict inequality for a nonconstant scale. An expectation equality therefore does not reproduce the normal interval's nominal coverage.

For an original strictly conservative example, let $R$ take $0.25$ and $2.75$ with equal probabilities. Then $E{\hat{V}}_n=1.5V_n$, but at nominal 95 percent the exact coverage isThis value is unchanged as $n$ grows. Meanwhile ${\hat{V}}_n-V_n=(R-1)/n$ tends to zero, showing why absolute consistency at a vanishing variance scale is insufficient. The normal cdf evaluation uses the pinned primary numerical definition.^[ref]

If the scale depends on the standardized error, the conditional normal formula above cannot simply be retained; the joint law is needed. NIST likewise distinguishes a justified coverage probability from selecting a conventional uncertainty multiplier.^[ref]

A samplewise variance upper bound is a nonnegative statistic $\hat{V}$ satisfying ${\mathrm{Pr}}_{\theta }\{\hat{V}\ge V_{\theta }\}=1$, where $V_{\theta }={\mathrm{Var}}_{\theta }(\hat{\theta })$ is the fixed sampling variance at parameter $\theta$. It controls each realized value outside a probability-zero exceptional set. This implies the expected-value condition for a conservative variance estimator when the expectation exists; the converse fails, as the parent article's counterexample shows. The primary sampling-design definition only asserts the weaker expectation condition.^[ref]

Suppose $\hat{\theta }$ is an unbiased estimator with finite variance, and let $0<\alpha <1$. Then Chebyshev's inequality and the samplewise comparison giveNo independence between $\hat{V}$ and the centre is needed. If $V_{\theta }=0$, unbiasedness makes the centre equal to $\theta$ almost surely, so coverage still holds. This proves confidence bounds from a samplewise variance bound; it does not justify replacing $1/\sqrt{\alpha }$ by a normal distribution critical value.

For an original bounded-data example, take $n=25$ independent and identically distributed measurements $X_i$ known to lie in $[0,10]$, and estimate their population mean by the sample mean. With $\mu =E{X}_i$, the inequality $X_i(10-X_i)\ge 0$ givesThe deterministic choice $\hat{V}=1$ is a samplewise variance upper bound for every such population. Thus $[\overline{X}-\sqrt{20},\overline{X}+\sqrt{20}]$ covers $\mu$ with probability at least $0.95$. The radius $\sqrt{20}\approx 4.472$ shows the price of distribution-free reasoning; the estimate may be much wider than a model-specific confidence interval.

Confidence bounds from a samplewise variance bound use a variance comparison that holds for the realized statistic, together with a valid tail inequality. The unbiased two-sided construction is proved in samplewise variance upper bound. A useful extension includes a separately justified bias budget: assumewhere $B\ge 0$ is a validated deterministic bound and the estimator has finite variance. For $0<\alpha <1$, defineIf $\theta$ lies outside this interval, the triangle inequality implies $|\hat{\theta }-E_{\theta }\hat{\theta }|>\sqrt{\hat{V}/\alpha }$. The samplewise bound and Chebyshev's inequality then give ${\mathrm{Pr}}_{\theta }(\theta \notin I)\le \alpha$. No independence between the centre and $\hat{V}$ is required. Zero true variance is covered because the centre then equals its expectation almost surely.

For an original measurement example in a fixed unit, suppose the reported estimate is $2.00$, a validated residual-bias bound is $B=0.20$, and a deterministic samplewise variance upper bound is $0.04$. With $\alpha =0.05$, the radius isso the interval is approximately $[0.9056,3.0944]$. The conclusion applies over repeated measurements under those assumptions. The particular realized interval is not assigned a frequentist probability that the fixed parameter is inside it. Replacing the radius by $1.96\sqrt{0.04}=0.392$ would discard both the bias allowance and the tail justification.

The bound $B$ is an assumption about the centre's possible systematic displacement; a guessed correction uncertainty is not automatically such a bound. NIST distinguishes the coverage interpretation from merely applying a conventional multiplier to a standard uncertainty.^[ref] If only an upper or lower endpoint is required, a one-sided Cantelli confidence bound with a bias budget uses a one-sided inequality rather than silently reusing a two-sided normal critical value.

A one-sided Cantelli confidence bound with a bias budget combines a validated bound on the estimator's systematic displacement with a one-sided variance inequality. Suppose $|E_{\theta }\hat{\theta }-\theta |\le B$, where $B\ge 0$ is deterministic, and a nonnegative statistic satisfies $\hat{V}\ge V_{\theta }={\mathrm{Var}}_{\theta }(\hat{\theta })$ almost surely. For $0<\alpha <1$, defineThen ${\mathrm{Pr}}_{\theta }(\theta \ge L)\ge 1-\alpha$. This is a lower confidence bound, with an unbounded upper side.

For the proof, put $Y=\hat{\theta }-E_{\theta }\hat{\theta }$. If $\theta <L$, the bias bound impliesFor positive $V_{\theta }$, Cantelli's inequality bounds this event by $V_{\theta }/(V_{\theta }+a^2)=\alpha$ at the last displayed threshold $a$. For zero variance, $Y=0$ almost surely and the conclusion follows directly. A random $\hat{V}$ may depend on the centre: the samplewise comparison is what supports the event inclusion.

For an original numerical bound, let $\hat{\theta }=2.0$, $B=0.2$, $\hat{V}=0.04$, and $\alpha =0.1$. The square-root term is $0.6$, so $L=1.2$ is a 90 percent lower bound under these assumptions. Reversing the inequality gives the separately valid upper bound $U=2.8$. Each has 90 percent one-sided coverage; reporting both endpoints together only guarantees 80 percent by the union bound. To guarantee a joint 90 percent interval this way, allocate error probabilities summing to $0.1$ across its two sides.

This construction complements the two-sided result in confidence bounds from a samplewise variance bound. It needs a bias bound and an almost-sure variance bound, not merely a conservative variance estimator in the expected-value sense. A coverage interpretation must follow the actual tail argument, as emphasized by NIST's discussion of expanded uncertainty.^[ref]