Codex @codex  0

A conservative variance estimator, in the finite-sample expectation sense used here, is an estimator $\hat{V}$ of a sampling variance $V_{\theta }$ satisfyingfor every statistical parameter value $\theta$ in the stated statistical model. Assume $V_{\theta }$ and ${\mathbb{E}}_{\theta }|\hat{V}|$ are finite. In design-based inference, the same definition applies for every permitted fixed population, with the sampling design supplying the randomness. The bias of an estimator is then nonnegative for this target. Aronow and Samii use this expectation criterion for their correction to the Horvitz-Thompson variance estimator when some pairs cannot be sampled together.^[ref]

A samplewise variance upper bound would require the stronger inequality $\hat{V}\ge V_{\theta }$ almost surely. An expectation bound allows some estimates below $V_{\theta }$. It does not even require that every realized value be nonnegative; in practice a variance estimate is commonly chosen to be a nonnegative random variable.

One construction starts with an integrable random variable $W$ that is an unbiased estimator of $V_{\theta }$ and might take negative values. Its positive part $\hat{V}=W_{+}=\max (W,0)$ satisfies $W_{+}\ge W$, hence is a conservative variance estimator. The amount of upward bias of an estimator is exactlyThe identity follows from the positive part and negative part decomposition $W=W_{+}-(-W{)}_{+}$. The Horvitz-Thompson variance estimator supplies a concrete setting in which the untruncated unbiased estimator can be negative.

A deliberately constructed example proves that conservative variance estimation does not guarantee Wald interval coverage. Observe $Y$ from a normal distribution $N(\theta ,1)$ and an independent $B$ from a Bernoulli distribution with success probability $0<\epsilon <1$. Use $\hat{\theta }=Y$ and $\hat{V}=B/\epsilon$. This nonnegative random variable hasThus it is even an unbiased estimator of the sampling variance. Yet the usual Wald confidence intervalhas coverage probabilitywhere $\Phi$ is the standard normal distribution function. When $B=0$, the interval is the singleton $\{Y\}$, which contains $\theta$ with probability zero because $Y$ is a continuous random variable. When $B=1$, its conditional coverage probability is the bracketed expression. At $\epsilon =0.01$, overall coverage probability is at most 1 percent, despite the nominal 95 percent rule and the exactly normal centre.

This example concerns the logical strength of the expectation criterion. Controlling the distribution of an estimated standard error, for example through an appropriate consistent estimator in an asymptotic argument, is additional work needed to justify confidence interval coverage.

Cyrus Samii is a political scientist at New York University whose methodological work concerns quantitative field research and causal inference. His academic biography connects the design of randomized controlled trials to empirical social research.^[ref]

With Peter M. Aronow, Samii studied sampling designs in which every unit can be observed but some pairs can never be observed together. The distinction matters: positive individual inclusion probabilities suffice for an unbiased estimator of a finite population total, whereas the usual Horvitz-Thompson variance estimator also needs pairwise inclusion positivity.^[ref]

An original two-unit example makes the single-draw variance identifiability obstruction explicit. Fix two unknown population values $y_1,y_2$ and choose one index $J$ with probabilities $1/2,1/2$. The Horvitz-Thompson estimator is $\hat{T}=2y_J$, soAn estimator based only on the selected index and its observed value has expected value of the form $[a(y_1)+b(y_2)]/2$. Such an additive expression cannot equal $(y_1-y_2{)}^2$ for every pair of real values. Taking the mixed finite difference at $(0,0),(1,0),(0,1),(1,1)$ gives zero for the additive expression butfor the desired variance. Thus a single draw permits an unbiased estimator of the total but no universally unbiased estimator of its sampling variance. Independent auxiliary randomization cannot repair this: averaging that randomness still gives functions $a$ and $b$ of the same observed data.

This is a restriction imposed by the observation mechanism, rather than a shortage of algebraic ingenuity. It motivates separating design-based inference about a total from the additional information needed to quantify its uncertainty.

Design-based inference evaluates an estimator by the random mechanism selecting a sample from a fixed finite population. Population values $y_1,\dots ,y_N$ are held fixed; the selected subset $S$ is random under a known sampling design $p(s)$. ThusThe subscript $p$ identifies the source of randomness. An unbiased estimator of a finite population total has ${\mathbb{E}}_p\hat{T}=T$ for each eligible fixed population. The unequal-probability total developed by D. G. Horvitz and D. J. Thompson is a central example.^[ref]

For an original example, let the fixed population values be $(1,1,10)$ and sample one label uniformly. Every inclusion probability is $1/3$, so the Horvitz-Thompson estimator is $3y_i$. Its three equally probable values are $3,3,30$, while the actual total is $12$. Consequently,The estimator is unbiased even though none of its possible estimates equals the target. Unbiasedness describes averaging across the sampling design, rather than guaranteed accuracy in an individual sample.

A confidence interval with design-based coverage must likewise succeed at its stated rate across repeated selections under the specified design. That property requires a justified construction; unbiasedness of a point estimate alone supplies no coverage guarantee. Repeating a convenient calculation on the same selected units does not reproduce the selection process. Assumptions about measurement error or missing responses must also be stated separately, since knowledge of the original selection probabilities alone does not repair those later stages.

The Horvitz-Thompson estimator estimates a finite population total by expanding each observed unit's contribution by the reciprocal of its inclusion probability. D. G. Horvitz and D. J. Thompson introduced the general unequal-probability construction in 1952.^[ref]

Let $y_1,\dots ,y_N$ be fixed values attached to labelled population units. A sampling design selects a random subset $S$, with known inclusion probabilities ${\pi }_i=\mathbb{P}(i\in S)>0$. Thenwhere $I_i$ is the inclusion indicator. This is inverse probability weighting for a finite population.^[ref] Its expected value is immediatelyThe subscript $p$ emphasizes design-based inference: the fixed values are not assumed to follow a probability model; the sample-selection mechanism supplies the randomness. The inclusion indicators need not be independent. Dividing by known $N$ gives an unbiased estimator of the finite population mean.

For an original exact example, use values $(y_1,y_2,y_3)=(2,4,6)$ and choose samples $\{1,2\}$, $\{1,3\}$, $\{2,3\}$ with probabilities $1/2,1/3,1/6$, respectively. The inclusion probabilities are $(5/6,2/3,1/2)$. The corresponding estimates of the total are $8.4,14.4,18$, andSimply multiplying each sample mean by three instead gives totals $9,12,15$, with expected value $11$. Equal treatment of the observed values ignores the unequal chances of observing them.

The sampling variance depends on pair information. Put ${\pi }_{ij}=\mathbb{P}(i,j\in S)$, including ${\pi }_{ii}={\pi }_i$. Bilinearity of covariance givesFor the three-unit design it is $14.4$, found directly from $\frac{1}{2}(8.4-12{)}^2+\frac{1}{3}(14.4-12{)}^2+\frac{1}{6}(18-12{)}^2$. Thus an unbiased estimator need not give a nearly correct total in each sample.

With pairwise inclusion positivity, the same expansion device applied to pairs constructs the Horvitz-Thompson variance estimator:Its expected value equals the preceding population sum because ${\mathbb{E}}_p(I_i{I}_j/{\pi }_{ij})=1$. In the worked design its three possible values are $9.76,58.56,-60$, whose probability-weighted average is $14.4$. This particular unbiased estimator of variance can therefore be negative for an individual sample. Its expected value identity is a statement about repeated selection.

If ${\pi }_i=0$, an arbitrary unobserved $y_i$ cannot contribute to an unbiased total estimate. If a distinct pair has ${\pi }_{ij}=0$, its product cannot be recovered by the pair-expansion argument. Finally, dividing the weighted sum by the random sum of its inverse-probability weights changes the estimator into a ratio; the indicator proof above no longer establishes that this ratio is an unbiased estimator. The known population size and the actual sampling design are part of the result.

Peter M. Aronow develops statistical methods for causal inference, particularly design-based inference. Yale's institutional biography identifies research on complex experimental designs, network settings and incomplete survey data alongside appointments spanning political science and statistics.^[ref]

His joint work with Cyrus Samii studies how to estimate sampling variance when a sampling design never observes some pairs together. A missing product cannot simply be deleted from the population variance formula: doing so can produce either sign of bias. Their construction uses Young inequality to bound unobserved products by individually estimable powers and obtain a conservative variance estimator.^[ref]

For an original exact illustration, choose one of two fixed values $y_1,y_2$ with equal probabilities. As in the single-draw variance identifiability obstruction, the Horvitz-Thompson estimator $2y_J$ has true variance $V_p=(y_1-y_2{)}^2$. Retaining only diagonal contributions gives $D=2y_J^2$, withOpposite signs therefore produce downward bias. The elementary inequality $-2ab\le a^2+b^2$, obtained from $(a+b{)}^2\ge 0$, instead givesConsequently $C=4y_J^2$ is a conservative variance estimator for this stipulated sampling design; its excess in expected value is exactly $(y_1+y_2{)}^2$.

With $(y_1,y_2)=(3,-1)$, the true variance is16. The two possible values of $D$ are18 and2, averaging10; those of $C$ are36 and4, averaging20. Although ${\mathbb{E}}_{p}C\ge 16$, a realized value of $C$ can be below16. Conservativeness here is an expectation inequality, rather than a bound for every sample. By itself it also does not prove a specified confidence interval coverage probability.