Independent and identically distributed random variables ($X_i\stackrel{\mathrm{iid}}{\sim }\nu$)

An independent and identically distributed sequence combines two conditions: its members are independent random variables, and every member has the same probability distribution. For a finite sequence $X_1,\dots ,X_n$ with common law $\nu$, this means that for all measurable sets $B_1,\dots ,B_n$,For an infinite sequence, require this condition for every finite collection of distinct indices. Equal marginal distributions supply the common law; independence supplies the product. Neither condition supplies the other. Grinstead and Snell call the combination an independent trials process.^[ref]

Choose a Bernoulli distribution with success probability $1/2$. Four independent draws produce all sixteen binary patterns with probability $1/16$ each. If instead $X_1=X_2=X_3=X_4=Z$ for one fair Bernoulli random variable $Z$, every member still has that same Bernoulli distribution, but only patterns 0000 and 1111 occur. Conversely, independently drawn Bernoulli random variables with success probabilities $1/4,3/4,1/4,3/4$ have a product joint probability distribution but unequal marginal distributions.

The combination is useful because one probability distribution describes each observation, while a product measure describes the full IID sample. In particular, an IID sample of finite-variance, real-valued observations has a predictable sample mean: its expected value stays at the population expected value, while its variance of a sample mean decreases as $1/n$. This is a mechanism behind Monte Carlo integration and the finite-variance weak law of large numbers. Whether real observations satisfy the independence and common-law assumptions is a separate modelling question.

For $n\ge 1$ independent and identically distributed Bernoulli random variables with success probability $p$, the sum $K={\sum }_i{X}_i$ has a binomial distribution. Every length-$n$ pattern with $k$ successes has probability $p^k(1-p{)}^{n-k}$, and the binomial coefficient counts its $\left(\genfrac{}{}{0ex}{}{n}{k}\right)$ possible success positions. ThereforeThe usual endpoint convention includes $p=0$ and $p=1$ as deterministic cases. For $p=1/2$ and $n=4$, the five probability masses are $1,4,6,4,1$ divided by $16$. The sample mean takes values $0,1/4,1/2,3/4,1$, with variance $1/16$ and standard deviation $1/4$. At $n=64$, its variance is $1/256$ and its standard deviation is $1/16$.

In the copied-coin model, $X_i=Z$ for every $i$, so the sample mean is exactly $Z$, however large $n$ is. Its expected value is $1/2$ and its variance is $1/4$, matching each observation's marginal variance. The off-diagonal covariances in the variance of a sum explain the difference:For an IID sample, those off-diagonal terms are zero. Repeating the same random input therefore does not create the reduction in fluctuations obtained from fresh independent draws. The general common-plus-individual structure is described by compound-symmetry covariance.

The IID sample property places no integrability condition on its common probability distribution. An IID sample with the standard Cauchy distribution has no finite expected value or variance. The Cauchy sample mean stability result shows that its averages keep that same Cauchy distribution for every sample size. Thus increasing $n$ need not concentrate an IID sample around a population mean that does not exist. The finite-variance bound below applies only when its hypotheses hold.

Draw two different labels uniformly from $\{1,2,3,4\}$ using sampling without replacement. Each position has a uniform distribution on a finite set, but $\mathbb{P}(X_1=X_2)=0$. Two independent random variables with those same marginal distributions would match with probability $1/4$. Equal marginal laws therefore do not make this sample an IID sample. Mark some of those four labels as successes. Their count in the two draws has a hypergeometric distribution; independent Bernoulli trials with a common success probability give a binomial distribution.