= Horvitz-Thompson variance estimator
{c}
{title2=$\widehat V_{\mathrm{HT}}$}
The \b[<Horvitz-Thompson variance estimator>] estimates the <sampling variance> of a <Horvitz-Thompson estimator> using observed values and known individual and <joint inclusion probabilities>. It is a random estimate of a fixed design-dependent <variance>, rather than the <variance> itself.
Let $I_i$ be an <inclusion indicator>, $\pi_i=\mathbb E_pI_i>0$, and $\pi_{ij}=\mathbb E_p(I_iI_j)$, with $\pi_{ii}=\pi_i$. For fixed population values $y_i$, write $a_i=y_i/\pi_i$. Bilinearity of <covariance> gives
$$
V_p=\operatorname{Var}_p\!\left(\sum_i I_i a_i\right)
=\sum_i\sum_j(\pi_{ij}-\pi_i\pi_j)a_i a_j.
$$
Under <pairwise inclusion positivity>, a sample $S$ provides the estimator
$$
\boxed{\widehat V_{\mathrm{HT}}=
\sum_{i\in S}\sum_{j\in S}
\frac{\pi_{ij}-\pi_i\pi_j}{\pi_{ij}}\frac{y_i}{\pi_i}\frac{y_j}{\pi_j}.}
$$
The diagonal terms are included. Taking <expected values> replaces $I_iI_j/\pi_{ij}$ by 1, proving $\mathbb E_p\widehat V_{\mathrm{HT}}=V_p$. This is the pairwise counterpart of inverse-probability expansion for the <finite population total>.<Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities by Peter M. Aronow and Cyrus Samii>[Section 2.1, variance expression and unbiased variance estimator]{ref}
An original exact example uses fixed values $(3,0,1)$. Select $\{1,2\},\{1,3\},\{2,3\}$ with probabilities $1/2,1/4,1/4$, respectively. Then $(\pi_1,\pi_2,\pi_3)=(3/4,3/4,1/2)$, and the possible total estimates are $4,6,2$. Their <expected value> is the true total 4, with <variance>
$$
\tfrac12(4-4)^2+\tfrac14(6-4)^2+\tfrac14(2-4)^2=2.
$$
For those same samples, the variance estimates are $4,-2,2$. Their <expected value> is also 2. In particular, for $S=\{1,3\}$, the two diagonal contributions are 4 and 2, while the combined off-diagonal contribution is $-8$, giving $-2$.
Thus this <unbiased estimator> of <variance> can be negative in a realized sample. Replacing a negative result by zero changes its <expected value>; in this example that modification raises the mean estimate to $5/2$. A positive square root is therefore not always available as a sample <standard error> from this particular formula. If some $\pi_{ij}=0$, dropping an unobservable cross-product does not preserve the proof; the missing term needs separate analysis.
Back to article page
