 Conservative variance estimator

A conservative variance estimator, in the finite-sample expectation sense used here, is an estimator $\hat{V}$ of a sampling variance $V_{\theta }$ satisfyingfor every statistical parameter value $\theta$ in the stated statistical model. Assume $V_{\theta }$ and ${\mathbb{E}}_{\theta }|\hat{V}|$ are finite. In design-based inference, the same definition applies for every permitted fixed population, with the sampling design supplying the randomness. The bias of an estimator is then nonnegative for this target. Aronow and Samii use this expectation criterion for their correction to the Horvitz-Thompson variance estimator when some pairs cannot be sampled together.^[ref]

A samplewise variance upper bound would require the stronger inequality $\hat{V}\ge V_{\theta }$ almost surely. An expectation bound allows some estimates below $V_{\theta }$. It does not even require that every realized value be nonnegative; in practice a variance estimate is commonly chosen to be a nonnegative random variable.

One construction starts with an integrable random variable $W$ that is an unbiased estimator of $V_{\theta }$ and might take negative values. Its positive part $\hat{V}=W_{+}=\max (W,0)$ satisfies $W_{+}\ge W$, hence is a conservative variance estimator. The amount of upward bias of an estimator is exactlyThe identity follows from the positive part and negative part decomposition $W=W_{+}-(-W{)}_{+}$. The Horvitz-Thompson variance estimator supplies a concrete setting in which the untruncated unbiased estimator can be negative.

A deliberately constructed example proves that conservative variance estimation does not guarantee Wald interval coverage. Observe $Y$ from a normal distribution $N(\theta ,1)$ and an independent $B$ from a Bernoulli distribution with success probability $0<\epsilon <1$. Use $\hat{\theta }=Y$ and $\hat{V}=B/\epsilon$. This nonnegative random variable hasThus it is even an unbiased estimator of the sampling variance. Yet the usual Wald confidence intervalhas coverage probabilitywhere $\Phi$ is the standard normal distribution function. When $B=0$, the interval is the singleton $\{Y\}$, which contains $\theta$ with probability zero because $Y$ is a continuous random variable. When $B=1$, its conditional coverage probability is the bracketed expression. At $\epsilon =0.01$, overall coverage probability is at most 1 percent, despite the nominal 95 percent rule and the exactly normal centre.

This example concerns the logical strength of the expectation criterion. Controlling the distribution of an estimated standard error, for example through an appropriate consistent estimator in an asymptotic argument, is additional work needed to justify confidence interval coverage.