Probability sampling chooses observational units by a specified random mechanism with known inclusion probabilities. This separates design-based representativeness from the convenience of obtaining particular observations. Simple random sampling, stratified sampling and cluster sampling are different designs with different precision and cost properties.
A finite population total adds a specified numerical value over every labelled unit in a fixed, finite population :The target must specify both the population and the measured variable. It may be a total inventory mass, the total number of employees across establishments, or the sum of a binary property. If , the total counts units having the property; it is not automatically the population size . A finite population mean is , when is known and positive.
An original inventory has four containers holding of a material. The total is and the finite population mean is per container. Giving every container the same mass fraction of an analyte is a separate assumption: the total analyte mass instead requires summing each container's mass multiplied by its own mass fraction. The target follows the physical quantity being added.
When only a sample is observed, the unweighted sample sum is generally not an unbiased estimator of . Using inclusion indicators and inclusion probabilities givesIf every , the Horvitz-Thompson estimator instead satisfiesThis is a consequence of the sampling design, without requiring the fixed population values to follow a statistical distribution.[ref]
A Generalization of Sampling Without Replacement From a Finite Universe by D. G. Horvitz and D. J. Thompson. 1952, finite-universe estimation framework, pp.663–685
A full census removes sample-selection variation, but imperfect measurement can still make the recorded sum differ from the intended physical total.Design-based inference evaluates an estimator by the random mechanism selecting a sample from a fixed finite population. Population values are held fixed; the selected subset is random under a known sampling design . ThusThe subscript identifies the source of randomness. An unbiased estimator of a finite population total has for each eligible fixed population. The unequal-probability total developed by D. G. Horvitz and D. J. Thompson is a central example.[ref]
A Generalization of Sampling Without Replacement From a Finite Universe by D. G. Horvitz and D. J. Thompson. 1952, unequal-probability sampling and unbiased finite-universe estimation
For an original example, let the fixed population values be and sample one label uniformly. Every inclusion probability is , so the Horvitz-Thompson estimator is . Its three equally probable values are , while the actual total is . Consequently,The estimator is unbiased even though none of its possible estimates equals the target. Unbiasedness describes averaging across the sampling design, rather than guaranteed accuracy in an individual sample.
A confidence interval with design-based coverage must likewise succeed at its stated rate across repeated selections under the specified design. That property requires a justified construction; unbiasedness of a point estimate alone supplies no coverage guarantee. Repeating a convenient calculation on the same selected units does not reproduce the selection process. Assumptions about measurement error or missing responses must also be stated separately, since knowledge of the original selection probabilities alone does not repair those later stages.
A sampling design specifies a probability distribution over the possible samples from a defined population:For sampling distinct labelled units, is a subset of the population. The design includes the selection mechanism, not merely the final sample size. A procedure with repeated draws can instead require a distribution over ordered sequences; the resulting inclusion probabilities concern whether a label appears at least once.
The design determines inclusion probabilities and joint inclusion probabilities throughThese connect the physical sampling procedure to the Horvitz-Thompson estimator and its sampling variance.[ref]
A Generalization of Sampling Without Replacement From a Finite Universe by D. G. Horvitz and D. J. Thompson. 1952, general unequal-probability sampling without replacement
An original comparison uses four fixed values and selects two units. Under simple random sampling, every one of the six pairs has probability . Each , so the Horvitz-Thompson estimator is twice the selected sum. It takes values with probabilities . Its expected value is the true finite population total , and its variance isA second design chooses only or , each with probability . It has the same sample size and the same individual inclusion probabilities, but the estimator is now or , giving variance . Pair selection matters even when individual representation agrees.
A third design chooses uniformly among the four pairs containing one label from each of those two groups. Its total estimate is always for this particular population. This illustrates the potential gain from stratified sampling when groups correspond to relevant variation, not a claim that the third design dominates for every possible population. A reproducible design records the eligible labels and actual random selection rule before interpreting the observed sample.
The Horvitz-Thompson estimator estimates a finite population total by expanding each observed unit's contribution by the reciprocal of its inclusion probability. D. G. Horvitz and D. J. Thompson introduced the general unequal-probability construction in 1952.[ref]
A Generalization of Sampling Without Replacement From a Finite Universe by D. G. Horvitz and D. J. Thompson. Original publication, Journal of the American Statistical Association 47(260), pp. 663–685, December 1952
Let be fixed values attached to labelled population units. A sampling design selects a random subset , with known inclusion probabilities . Thenwhere is the inclusion indicator. This is inverse probability weighting for a finite population.[ref]
Unequal Probability Sampling in Penn State STAT 506. Lesson 3, Definition 3.2, Horvitz–Thompson estimator
Its expected value is immediatelyThe subscript emphasizes design-based inference: the fixed values are not assumed to follow a probability model; the sample-selection mechanism supplies the randomness. The inclusion indicators need not be independent. Dividing by known gives an unbiased estimator of the finite population mean.For an original exact example, use values and choose samples , , with probabilities , respectively. The inclusion probabilities are . The corresponding estimates of the total are , andSimply multiplying each sample mean by three instead gives totals , with expected value . Equal treatment of the observed values ignores the unequal chances of observing them.
Inverse-inclusion weighting corrects the expected total in an unequal-probability sample
. Original exact enumeration of three possible two-unit samples from values 2, 4 and 6. The left panel shows their selection probabilities; the right compares the Horvitz-Thompson total with three times the sample mean. The population total is 12. Details.The sampling variance depends on pair information. Put , including . Bilinearity of covariance givesFor the three-unit design it is , found directly from . Thus an unbiased estimator need not give a nearly correct total in each sample.
With pairwise inclusion positivity, the same expansion device applied to pairs constructs the Horvitz-Thompson variance estimator:Its expected value equals the preceding population sum because . In the worked design its three possible values are , whose probability-weighted average is . This particular unbiased estimator of variance can therefore be negative for an individual sample. Its expected value identity is a statement about repeated selection.
If , an arbitrary unobserved cannot contribute to an unbiased total estimate. If a distinct pair has , its product cannot be recovered by the pair-expansion argument. Finally, dividing the weighted sum by the random sum of its inverse-probability weights changes the estimator into a ratio; the indicator proof above no longer establishes that this ratio is an unbiased estimator. The known population size and the actual sampling design are part of the result.
The Horvitz-Thompson variance estimator estimates the sampling variance of a Horvitz-Thompson estimator using observed values and known individual and joint inclusion probabilities. It is a random estimate of a fixed design-dependent variance, rather than the variance itself.
Let be an inclusion indicator, , and , with . For fixed population values , write . Bilinearity of covariance givesUnder pairwise inclusion positivity, a sample provides the estimatorThe diagonal terms are included. Taking expected values replaces by 1, proving . This is the pairwise counterpart of inverse-probability expansion for the finite population total.[ref]
An original exact example uses fixed values . Select with probabilities , respectively. Then , and the possible total estimates are . Their expected value is the true total 4, with varianceFor those same samples, the variance estimates are . Their expected value is also 2. In particular, for , the two diagonal contributions are 4 and 2, while the combined off-diagonal contribution is , giving .
Thus this unbiased estimator of variance can be negative in a realized sample. Replacing a negative result by zero changes its expected value; in this example that modification raises the mean estimate to . A positive square root is therefore not always available as a sample standard error from this particular formula. If some , dropping an unobservable cross-product does not preserve the proof; the missing term needs separate analysis.
The single-draw variance identifiability obstruction is the inability, for an unrestricted finite population, to construct a universally design-unbiased estimate of a total estimator's variance from just one observed unit. The estimator can estimate the total unbiasedly while the required cross-products for estimating its sampling variance are never jointly observed. This is an obstruction to the stated one-draw estimation problem, not a claim that a parameter can never be identified from repeated independent samples.
An original two-unit proof makes the distinction exact. Sample label or2 with equal probability and observe only . The Horvitz-Thompson estimator of the finite population total is . Its expectation is , and its design variance isSuppose a proposed variance estimate returns after observing unit1 and after observing unit2. Unbiasedness for every pair would requireThe left side is additive in the two unknown values, while the right side contains an unobserved product.
To prove impossibility without smoothness assumptions, evaluate both sides at . Every additive function has rectangular contrast . The required variance has contrast . The contradiction rules out such an unbiased estimator. Independent auxiliary randomization does not help: averaging it out merely replaces each by another function of the same single observed value.
Here the individual inclusion probabilities are positive, but the joint inclusion probability for distinct units is zero. The usual Horvitz-Thompson variance estimator needs pairwise inclusion positivity to turn sample cross-products into unbiased population cross-product estimates. The primary analysis of zero pairwise probabilities identifies precisely these missing terms.[ref]
Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities by Peter M. Aronow and Cyrus Samii. §2.1, pair-expansion variance estimator; §2.2 and Proposition1, missing products under zero pairwise probabilities
Extra information can change the problem: observing suitable pairs, imposing a justified population relation, or accepting a model-based or conservative estimate uses assumptions absent from the proof. Observing one value also does not prevent writing a biased variance estimate. What fails is a single-draw unbiased estimator valid for every unrestricted population, even though the total estimator itself remains unbiased.
Sampling with replacement returns each selected label to the available population before the next draw. Independent uniform draws from a fixed population are independent and identically distributed; a label can recur. Replacement by itself does not imply uniformity or independence.
An inclusion probability is the probability that a particular unit belongs to the selected sample. For sample size in simple random sampling from labels it is . It differs from the selection probability at a particular draw position.
A joint inclusion probability is the probability that two specified population units both belong to a random sample :For the same unit twice, , where is its inclusion probability. For distinct units, is generally not : selection can make their inclusion indicators dependent. These probabilities describe the sampling procedure, rather than a relationship between the measured values of the two units.[ref]
Unequal Probability Sampling in Penn State STAT 506. Lesson 3, inclusion probabilities and finite-population estimation
For simple random sampling of distinct labels from labels, counting subsets containing both and giveswith value zero if . Since ,For , each unit has inclusion probability , but a particular pair has joint inclusion probability . Conditional on including one member, the other is included with probability , not .
Equal individual inclusion probabilities do not determine these pair probabilities. A different procedure that chooses only or , each with probability , still includes each unit with probability . Yet and . Thus two procedures can agree on individual representation while producing different dependence and different precision for a population summary. The distinction motivates keeping pair information when calculating the variance of weighted survey estimators; the classical general treatment is due to D. G. Horvitz and D. J. Thompson.[ref]
A Generalization of Sampling Without Replacement From a Finite Universe by D. G. Horvitz and D. J. Thompson. Publisher abstract and original publication, pp. 663–685
Pairwise inclusion positivity requires every distinct pair of population units to have positive joint inclusion probability under the sampling design:For Horvitz-Thompson variance estimator, the probabilities must also be known, and individual inclusion probabilities must be positive. Each pair can then, at least in principle, contribute an observed product to a pair-expansion estimate. Positivity is not independence: need not equal .[ref]
Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities by Peter M. Aronow and Cyrus Samii. Section 2, conditions for measurability; section 2.1, pair expansion
For simple random sampling of distinct labels from , . Consequently, gives pairwise positivity, whereas selecting exactly one label gives positive individual inclusion probabilities but zero probability of ever observing two distinct values together. This distinction separates unbiased total estimation from unbiased estimation of its sampling variance.
An original two-unit proof shows what information is missing. Sample one of two fixed values uniformly. The Horvitz-Thompson estimator is or , and its variance isAny estimator based only on the observed label and value has expected value of the form . Its mixed difference over is zero. The target's mixed difference is insteadNo such estimator can therefore be unbiased for this variance for every possible pair of values. Additional randomization internal to the estimator does not help: conditioning on the observed data still produces functions of this same separated form. This is an information limitation, not division by zero that can be repaired by assigning a convenient numerical convention.[ref]
Conservative variance estimation for sampling designs with zero pairwise inclusion probabilities by Peter M. Aronow and Cyrus Samii. Sections 1 and 2.2, impossible unbiased recovery of jointly unobservable cross-products
Strict positivity alone also does not guarantee useful precision. A very small produces a large inverse-probability multiplier on a rare observed pair. Designing a usable survey therefore requires attention to both observability and the resulting sampling variance.
An inclusion indicator records whether a specified population unit appears in a sample :It is an indicator random variable, with expected value and variance . Its randomness comes from sample selection; the population value attached to unit may be treated as fixed in a sampling calculation.[ref]
Unequal Probability Sampling in Penn State STAT 506. Lesson 3, inclusion probabilities and inverse-probability total estimation
Inclusion indicators turn a sum over a changing sample into a sum over fixed population labels:If every , the same identity immediately provesThe inverse-inclusion-probability total is unbiased because , not because the selected units must look typical in every realization.
Pair products connect the notation to joint inclusion probabilities: , so . For a fixed sample size , in every realization. Consequently,In simple random sampling with , the four individual variances sum to , while the six distinct pair covariances are each . Their doubled sum is , giving total variance zero. Treating the inclusion indicators as independent would incorrectly predict a random sample size. The notation makes that error easy to detect.
Unequal probability sampling assigns different inclusion probabilities to different population units. Estimating an unweighted population summary generally requires accounting for these differences in the estimator.
The finite population mean for is the arithmetic mean of all fixed values in the target population. Under simple random sampling, the sample mean is an unbiased estimator of it.
For , the finite population variance measures spread about the finite population mean. Survey-sampling formulas commonly use denominator ; the denominator- version is . The normalization must be specified when using a finite population correction.
Sampling without replacement selects distinct labelled units: after a unit is selected it is unavailable for the remaining draws. Distinct labels can carry equal numerical values, so repeated values do not necessarily imply a repeated unit. The rule alone does not specify selection probabilities; a procedure favoring particular units is still sampling without replacement but need not be simple random sampling.
In uniform sequential sampling without replacement, the next label is chosen uniformly among those remaining. For a population of labels, any ordered sequence of distinct labels has probability . Each size- subset has possible orders, so its probability is . This gives the fixed-size simple random sampling design. Equivalently, take the first labels of a uniformly random permutation. Restricting a uniformly random permutation to a specified label set preserves uniformity; biased modulo mappings should not replace uniform selection.
The draw positions have identical marginal probability distributions, but exhibit statistical dependence. If the first label is , the conditional probability that the second label is is zero, whereas its marginal probability is . Thus these draws are not independent and identically distributed when and . For a marked subset of labels, counting size- subsets with binomial coefficients gives the hypergeometric distribution. The resulting negative covariance explains the finite population correction for both counts and sample means.
Draw two distinct positions uniformly from a fixed population with . Let and . Each draw has expected value , whileSubtracting the product of the expected values proves the covariance formula and explains the finite population correction. If the sampled values are coin biases and the actual tosses are conditionally independent given those biases, the law of total covariance gives the same covariance for their head indicator random variables.
Take a uniformly random permutation of distinguishable objects, of which are marked. Let mean that position is marked. Every object is equally likely at a specified position, so . For distinct positions , their ordered pair is uniform among distinct-object pairs, of which are marked pairs. This proves the second formula. For , the conditional probability is .
For the count of marked objects among the first positions, indicator random variables and linearity of expectation give . These draws are exchangeable but generally dependent; equal marginal probabilities do not imply independence.
Draw balls sequentially without replacement from green and red balls. Let be the first positive draw count at which the numbers of drawn green and red balls agree. For ,where denotes the falling factorial and is a Catalan number. Odd first return times are impossible because equality requires an even total number of draws.
To count the successful colour orders, encode green by a step and red by a step . If the first step is positive, a first return time means that the partial sums remain strictly positive until the last step. Delete the first positive step and the last negative step, and shift the remaining heights down by one. The result is a Dyck path of length , of which there are . Interchanging the colours doubles this count. Every such colour order has the same probability under sampling without replacement, namely , proving the formula. In particular, the numbers of successful colour orders at lengths two, four and six are two, two and four; balanced orders with an earlier return must not be counted.
When items among are marked, inspect a uniformly random permutation until the first marked item. For its first position to be , the other marked positions must lie among the last slots. This yields the displayed law with support . It is the first-success form of sampling without replacement. For one marked item the position is uniform, while its conditional success chance after failures increases to .
Cluster sampling chooses groups of observational units, such as production days, rather than dispersed individual units. Testing every unit in a selected group is a one-stage cluster sample. Positive within-cluster intraclass correlation reduces the information supplied by a fixed number of sampled units, although collecting clustered observations can be cheaper.
The design effect is a sampling variance under a complex design divided by the corresponding simple-random-sampling variance. For independent equal-size clusters of size with exchangeable intraclass correlation , a common mean-estimation approximation is . It expresses the loss of effective sample size from positive correlation; finite-population and unequal-cluster-size designs require their own calculation.
Stratified sampling partitions a population into strata and samples separately within each. It guarantees specified representation and can improve precision when the strata describe population variation. Equal sampling fractions give equal unit inclusion probabilities; unequal fractions require appropriate weighting for population summaries.
Suppose the variance is and the sampling cost is , with positive costs, variance coefficients and budget. The Cauchy-Schwarz inequality gives . Equality forces proportional to and a tight budget, proving the displayed unique optimum. The minimal variance is . For the optimization Lagrangian , the multiplier and budget derivative both equal . Therefore a small resource change changes the optimum by . This is a continuous allocation; integer sample-size restrictions define a separate optimization problem.
Here simple random sampling means a random unordered subset of exactly distinct labels from , with every size- subset equally likely. The sample size is fixed in advance, , and sampling without replacement prevents duplicate units. This is SRSWOR; some texts also use "simple random sampling" for independent uniform draws with replacement, which is a different design.[ref]
Chapter 2: Simple Random Sampling. Sampling Theory, Chapter 2, Simple Random Sampling, PDF pp. 1–4: replacement conventions and ordered versus unordered sample probabilities
Every label has inclusion probability , but equal inclusion probabilities alone are insufficient. For example, from six labels choose either or with probability . Each label is included with probability , yet eighteen of the twenty size-three subsets are impossible: this is a two-group cluster sampling design. Unequal probability sampling deliberately changes inclusion probabilities, often to improve precision for a particular estimand, and generally requires an appropriately weighted estimator rather than the plain sample mean.
Consider the original mathematical population with labelled values , and let . The fifteen equiprobable subsets, followed by their sample means, areTheir sample means sum to , giving expected value . Their squared deviations from sum to , giving variance . This is an enumeration over hypothetical selections from six stipulated numbers, not measured survey data.
All fifteen size-two subsets and all thirty-six ordered replacement outcomes from six labelled values
. Details. With independent uniform sampling with replacement, the ordered outcomes each have probability . The two orders and are distinct outcomes, and is now possible. Forgetting order does not make the resulting unordered multisets equiprobable: has probability but has probability . The thirty-six sample means still have expected value , but their squared deviations sum to , giving variance . Avoiding duplicate labels improves precision for this fixed population.
In finite-population sampling, a simple random sample of fixed size without replacement is a subset of labelled units drawn so that every size- subset has probabilityThe sample is the selected subset; simple random sampling is the procedure that generates it. This definition does not require the observed values to be distinct, only the selected population labels. The without-replacement convention is important because an independent repeated draw from a population is a different design.[ref]
Unequal Probability Sampling in Penn State STAT 506. Lesson 3, simple-random-sampling specialization
One construction randomly orders all labels and takes the first . Each particular subset can occupy those positions in permutations, out of , proving the probability above. Equivalently, draw successively and uniformly from the remaining labels, then discard their order.
For , there are possible samples. A particular unit belongs to four of them, so its inclusion probability is ; a particular pair is one complete sample, so its joint inclusion probability is . These are properties of the procedure before observing which subset was obtained.
A simple random sample need not resemble the whole population in every realization. For fixed values , six of the ten size-two samples contain only zeros. Thus the sample mean is zero with probability , although its expected value is the finite population mean . The remaining four samples each have mean , giving . Unbiasedness concerns repeated selection, rather than a guarantee about any one sample. Coverage of the population labels and faithful implementation of the selection rule remain essential.
Fix real population values . Define the finite population mean and two normalizations of the finite population variance byThe values are fixed; only the selected subset is random. For its sample mean , the exact design expectation and variance areThe sample mean is an unbiased estimator of the finite population mean. At , every unit is observed and exactly, so the variance is zero. The case is deterministic and is handled directly, since is undefined then.[ref]
Chapter 2: Simple Random Sampling. Sampling Theory, Chapter 2, PDF pp. 2, 6–9: variance normalizations, unbiasedness, covariance derivation and finite population correction
The finite population correction depends on the baseline. With the normalization it is multiplying . With the normalization it is multiplying , the exact variance for independent uniform sampling with replacement. The standard error uses the square root of the relevant variance factor. Mixing the two normalizations gives a wrong answer even at , when replacement cannot matter. For the six-value example, , , and , reproducing without replacement and with replacement.
A short proof exposes the statistical dependence. Let be the value at position in uniform sequential sampling without replacement, so each has expected value and variance . Write . Every ordered pair of distinct labels is equally likely, and , hence for ,Linearity of expectation proves unbiasedness without requiring independence. The variance of a sum includes all ordered covariance terms, givingFor the original population, each pair of distinct draw positions has covariance . Drawing a high value leaves a lower mean among the remaining values, balancing subsequent draws. Equal marginal probability distributions therefore coexist with negative covariance.
When , the sample variance is an unbiased estimator of . To check this independently, use : its expected value is . Thus unbiasedly estimates the variance of the sample mean; its square root estimates the standard error, but need not be unbiased.[ref]
Chapter 2: Simple Random Sampling. Sampling Theory, Chapter 2, PDF pp. 10–12: sample-variance and standard-error estimation
Exact mean variance and expected distinct-unit coverage as sample size grows in the six-value population
. Details. The diagram compares variance with the expected fraction of distinct population labels observed. Without replacement this fraction is exactly ; with independent uniform sampling with replacement, a label is missed with probability , so linearity of expectation gives expected fraction . At six draws, replacement covers only about of labels on average and retains sample mean variance ; sampling all six distinct labels covers the population completely and removes selection uncertainty. This population coverage is distinct from confidence interval coverage probability.
The correction matters when the sampling fraction is appreciable. It calibrates uncertainty about this fixed population under the specified selection mechanism. It does not correct unequal selection weights, missing observations or measurement errors; observing every label also does not eliminate uncertainty about a larger unobserved population. Moreover, an exact variance alone does not guarantee the coverage probability of a proposed confidence interval: a small discrete sampling distribution can make a normal approximation inaccurate.
The sampling fraction is the sample size divided by the finite population size. In simple random sampling, it determines how much the finite population correction reduces the variance relative to .
Articles by others on the same topic
There are currently no matching articles.


