Population genetics

Population genetics studies the frequencies of alleles and genotypes in populations, and their changes through inheritance, mutation, selection, migration and random sampling. Kingman's coalescent describes a neutral sample's ancestry backward in time.

An allele changes in frequency because of association with another selected allele. Recombination can weaken that association, so the response differs between linked and freely recombining backgrounds.

The counts or proportions of segregating variants at different sample frequencies. Unfolded spectra use an inferred ancestral state; folded spectra use minor-allele counts.

A comparison of nonsynonymous and synonymous polymorphism and divergence. Under a simple neutral model their ratios match. An excess of nonsynonymous divergence can support adaptive fixation, but mildly deleterious polymorphism, changing demography and site-class assumptions affect interpretation.^[ref]

Nonrandom distribution of genetic variation among populations, shaped by ancestry, movement, reproduction and drift.

Increasing genetic differentiation with geographic separation when gene flow tends to occur over shorter distances.

The folded site frequency spectrum counts variable sites by the less frequent allele count, without identifying which allele is ancestral. With four sampled sequences, counts one and three both contribute to the $3:1$ class, whereas count two gives the $2:2$ class.

A measure of genetic association between individuals relative to a reference population. Simple pedigree values assume specified inheritance and mating patterns; relatedness relevant to selection is not merely the fraction of genes two individuals share.

Stochastic changes in lineage frequency without an imposed fitness difference. In a renewing stem-cell compartment, equivalent clones can be lost or expand while the total population remains approximately constant.

A genome-sequencing project sampling diverse human populations and informing models of shared and differentiated population ancestry.

Mixing of ancestry through interbreeding between previously differentiated populations. A simple branching tree may not represent the resulting history.

A framework tracing sampled genetic lineages backwards to shared ancestry. A gene tree coalescence time is not generally identical to a population separation or migration time.

Stochastic loss of the least mutation-loaded class in a finite population with little or no genetic recombination and negligible reversal of the harmful changes. Recombination can recreate some low-load combinations.

Reduced efficacy of natural selection caused by associations between selected genetic loci in finite populations. genetic recombination can relieve some of this interference by changing the associations.

A statistic describing differentiation among biological populations relative to within-population variation. High $F_{ST}$ can reflect reduced diversity within a population; it does not alone establish adaptive sequence divergence.

A change in linked variation accompanying the spread of a favorable allele. Its genomic signature depends on ancestry, recombination rate and whether selection acts on a new or pre-existing variant; a population bottleneck can mimic some features.

Competition among independently arising beneficial lineages in a population with little or no genetic recombination. A favorable mutation can be lost because a competing lineage spreads more successfully.

A parameter measuring a relative reproductive advantage or disadvantage under a specified convention. For a haploid variant with relative genotype fitness $1+s$, positive $s$ is advantageous; other conventions require translation before comparison.

The probability distribution of increases in genotype fitness among beneficial mutations in a specified genetic background and environment. Weighting arrival effects by fixation probability changes the distribution among successful substitutions.

Inherited variation already present before a focal environmental change or episode of natural selection. Reuse of an existing allele differs historically from independent origin through a new mutation.

A sequence of changes between genotypes or phenotypes under natural selection. A model with one substitution at a time is an approximation that can fail when many beneficial lineages compete.

An inherited sequence change that spreads to allele fixation because of a favorable effect on genotype fitness in the relevant context. The distribution of effects among substitutions differs from that among newly arising mutations.

A model in which a phenotype is a point in a multidimensional trait space and genotype fitness declines with displacement from an optimum. With displacement $\mathbf{z}$ and mutational step $\mathbf{a}$, improvement in squared distance requires $2\mathbf{z}\cdot \mathbf{a}+\Vert \mathbf{a}{\Vert }^2<0$.

A heterozygote has greater genotype fitness than either corresponding homozygote. In the deterministic random-mating two-allele viability-selection model this gives an attracting interior allele frequency, unlike the repelling interior frequency produced by underdominance.

Underdominance means that the heterozygote has lower genotype fitness than both corresponding homozygotes. In a deterministic randomly mating biallelic population, it produces an unstable interior allele-frequency threshold separating the two fixation outcomes, unlike heterozygote advantage.

With homozygote fitness advantages $a,b>0$ over the heterozygote, the allele frequency $u$ of the first allele has an unstable threshold $b/(a+b)$. Below it that allele decreases; above it it increases. In a discrete viability-selection generation the same numerator gives $u^{\prime }-u$ after division by mean genotype fitness. Equal homozygote advantages give $\dot{u}=su(1-u)(2u-1)$, with stable endpoints and unstable midpoint.

Genotype fitness measures the reproductive or viability success assigned to a genotype in a specified selection model. Relative fitnesses weight Hardy-Weinberg proportions after viability selection; continuous-time fitness differences instead act as selection-rate differences. The convention and time units must be specified before comparing selection with migration.

An ancestral recombination graph records ancestral coalescences and genetic recombination events for sampled chromosomal material. Backwards in time, each pair of ancestral lineages merges at rate one in standard diploid coalescent units, while a lineage splits at a recombination breakpoint at rate $\rho /2$ times the genetic span of the material it carries. The breakpoint splits that lineage's ancestral material into two parental contributions. Waiting times are exponential with the sum of the current event rates. At each genomic position, retaining the ancestry at that position gives a marginal phylogenetic tree with the neutral Kingman's coalescent distribution. Different positions have dependent trees because they share parts of the graph.

// Target: biology.bigb

A population bottleneck is a temporary large reduction in population size. It changes ancestral coalescence rates and can leave fewer ancestral haplotypes and increased linkage disequilibrium after the population expands.

// Target: biology.bigb

Genetic drift is random variation of allele frequencies due to finite-population reproduction. In the neutral ancestry description, randomly sampled gene copies merge into common ancestors according to Kingman's coalescent.

// Target: probability-and-statistics.bigb

Change in allele frequencies when a population is established by a restricted sample of ancestors. Subsequent growth does not restore variation that was not sampled.

Effective population size is the size of an idealized population giving the relevant rate of genetic drift or ancestral coalescence in the population being modeled. In the standard neutral diploid coalescent, time is measured in $2N_e$ generations and the population-scaled mutation parameter is $\theta =4N_e\mu$.

// Target: biology.bigb

A segregating site is a position at which at least two states occur in the sampled chromosomes. Under an infinite sites mutation model, mutations below the sample's common ancestor correspond one-to-one to segregating sites. Their number is commonly denoted $S$.

For a neutral sample of size $n\ge 2$ under the infinite sites mutation model, the Watterson estimator is ${\hat{\theta }}_W=S/a_{n-1}$, where $a_{n-1}={\sum }_{i=1}^{n-1}1/i$. Since the coalescent has $ES=\theta a_{n-1}$, this estimator is unbiased. It matches the observed segregating-site count to its expected value, but need not maximize the exact segregating-site likelihood under the neutral coalescent.

For fixed $\theta >0$, a geometric mutation count in a coalescent epoch with mean ${\mu }_i=\theta /i$ has third moment ${\mu }_i+6{\mu }_i^2+6{\mu }_i^3$. The centered absolute third moment is at most $4\mathbb{E}[Y_i^3]+4{\mu }_i^3$, so its sum is $O(\log n)$. The total variance is asymptotic to $\theta \log n$, making the Lyapunov condition with $\delta =1$ hold. Rescaling by the harmonic sum gives the displayed limit. At $\theta =0$ the estimator is identically zero and the limit is degenerate. The variance of the Watterson estimator decreases only logarithmically with sample size at a fixed nonrecombining locus.

The moments of the mutation count on a neutral coalescent tree give this variance for ${\hat{\theta }}_W=S/H_{n-1}$. Since $H_{n-1}\sim \log n$ and the sum of inverse squares is bounded, it tends to zero. Together with unbiasedness, this gives mean-square convergence to the mutation parameter, so it is a consistent estimator.

The infinite-sites model assumes that every mutation occurs at a previously unmutated site, so each mutation on the sample genealogy produces a distinct segregating site. If mutation rate per lineage is $\theta /2$ in standard coalescent time units, a tree of total length $L$ has a Poisson distribution of mutation count with mean $\theta L/2$.

For $n\ge 2$, the probability generating function of the segregating-site count conditional on mutation parameter $\theta$ isThis follows by integrating the Poisson generating function over the independent exponential epoch lengths. With $W=L/2$, its density is $(n-1)e^{-w}(1-e^{-w}{)}^{n-2}$. Expanding the binomial factor and integrating $w^k{e}^{-(\theta +r+1)w}$ givesFor $k=0$ use ${\theta }^0=1$, including at $\theta =0$. A posterior density divided by a positive prior density is proportional to this likelihood, allowing posterior simulation output to estimate its maximizer.

Inbreeding is reproduction between genetically related individuals. It increases the chance that an offspring's maternal and paternal copies at a locus are identical by descent. Such homozygosity by descent is autozygosity.

Reduced reproductive or survival performance associated with mating among relatives or self-fertilization in the relevant comparison. Measuring seed set alone can miss later offspring costs.

The inbreeding coefficient is the probability that the two allele copies at a locus within an individual are identical by descent. It equals the kinship coefficient of the individual's parents. It concerns ancestral identity, rather than merely homozygosity of observed allelic states.

// Target: biology.bigb

For a biallelic locus with frequencies $p=1-q$ and $q$, and inbreeding coefficient $F$, ancestral-copy identity with probability $F$ gives genotype probabilities $P(AA)=p^2+Fpq$, $P(Aa)=2pq(1-F)$ and $P(aa)=q^2+Fpq$. In the IBD component both copies equal a single ancestral draw; in the remaining component the two ancestral alleles are independent population draws. A fully penetrant recessive disease therefore has risk $q^2+Fq(1-q)$.

// Target: biology.bigb

For an otherwise outbred pedigree, each of the two shared grandparents contributes $(1/2{)}^{2+2+1}=1/32$ to the kinship coefficient of first cousins. Thus their child has inbreeding coefficient $1/16$. Equivalently, cousin kinship is one quarter of the $1/4$ kinship of their sibling parents.

// Target: biology.bigb

Linkage disequilibrium means that knowing an allele at one genetic locus changes the probability of an allele at another genetic locus in population haplotypes. It compares their joint probability distribution with the product of their marginal distributions. This is useful because a genetic marker can carry information about another genetic locus: tag SNPs exploit this redundancy, without establishing a causal genetic association.

Consider two genetic loci with alleles $A,a$ and $B,b$. Sample one haplotype, rather than an unphased diploid genotype. Write its four haplotype frequencies as $f_{AB},f_{Ab},f_{aB},f_{ab}$, summing to one, and set the marginal allele frequencies to $p=f_{AB}+f_{Ab}$ and $q=f_{AB}+f_{aB}$. The linkage disequilibrium coefficient isThe second expression follows by substituting $f_{ab}=1-f_{AB}-f_{Ab}-f_{aB}$. The entire two-locus joint probability distribution is thenEvery row and column sum is unchanged when $D$ varies. Linkage equilibrium means $D=0$: all four haplotype frequencies equal products of allele frequencies, so the two allele indicators are independent random variables. For $p>0$, the conditional probability $P(B\mid A)=q+D/p$ shows directly how positive $D$ enriches $B$ among copies carrying $A$, while negative $D$ depletes it. Positive $D$ also enriches $ab$; negative $D$ enriches $Ab$ and $aB$. Swapping the labels $A$ and $a$ reverses the sign of $D$, so neither sign is intrinsically favorable.

For example, fix the allele frequencies at $p=0.60,q=0.40$. At linkage equilibrium, the haplotype frequencies in the order $AB,Ab,aB,ab$ are $(0.24,0.36,0.16,0.24)$. Positive linkage disequilibrium $D=0.12$ instead gives $(0.36,0.24,0.04,0.36)$, with $P(B\mid A)=0.60$; negative linkage disequilibrium $D=-0.12$ gives $(0.12,0.48,0.28,0.12)$, with $P(B\mid A)=0.20$. Counting each allele separately would miss the difference.

Nonnegative haplotype frequencies constrain $D$. Define the nonnegative bounds $L=\min (pq,(1-p)(1-q))$ and $U=\min (p(1-q),(1-p)q)$. The four entries above giveThus $|D|\le 1/4$ globally, but a particular pair of allele frequencies usually permits a smaller, asymmetric interval. The standardized linkage disequilibrium convention used here retains the sign:This convention requires $0<p,q<1$; if either genetic locus has only one allele in the population, $D=0$ but normalized measures are undefined. Some presentations report only $|D^{\prime }|$. At $D^{\prime }=1$ or $-1$, at least one of the four haplotypes has zero frequency; this alone does not prove that genetic recombination has never occurred.

For the indicator random variables $X=1_A$ and $Y=1_B$, their covariance is $D$, and their variances are $p(1-p)$ and $q(1-q)$. The allelic correlation measure is their squared correlation coefficient,The subscript distinguishes it from the recombination fraction $r$ in linkage-disequilibrium decay under recombination. Squaring loses the association's sign. In the illustrated positive and negative examples, $r_{\mathrm{LD}}^2=0.25$ in both, whereas $D^{\prime }=0.75$ and $-0.50$, respectively. For another original example, take $p=0.10,q=0.50,f_{AB}=0.10$: then $D=0.05,D^{\prime }=1$, yet $r_{\mathrm{LD}}^2=1/9$. Every $A$ carries $B$, but most copies carrying $B$ lack $A$. The frequency constraints and the distinction between these normalizations are analyzed in Mathematical properties of the r² measure of linkage disequilibrium by Jenna M. VanLiere and others (sections 2.1 and 2.5, pp. 131–132 and 135–136).

Genetic linkage describes transmission through meiosis; linkage disequilibrium describes a population's haplotype frequencies. Nearby genetic loci on a chromosome can be in linkage equilibrium. Conversely, mixing populations can create linkage disequilibrium even between genetic loci on different chromosomes. For example, mix equal numbers of independently sampled haplotypes from two populations, each internally at linkage equilibrium, with $p=q=0.80$ in one and $p=q=0.20$ in the other. The pooled allele frequencies are $p=q=0.50$, but $f_{AB}=(0.64+0.04)/2=0.34$, giving $D=0.09$. This is a consequence of population stratification, not a measurement of the recombination fraction.

Measured haplotype frequencies also depend on the sample. Finite sample size, related individuals, genetic-study ascertainment, genotyping error and uncertain haplotype estimation can change an estimate. A missing rare haplotype can give empirical $|D^{\prime }|=1$ without demonstrating its absence from the population. Report the sample definition and allele frequencies alongside the association measure; a nonzero sample estimate alone does not establish nonzero population linkage disequilibrium.

A concrete dataset is the 2005 A haplotype map of the human genome by The International HapMap Consortium. The International HapMap Consortium used genotyping platforms including Illumina BeadArray to obtain SNP genotype calls from 269 DNA samples across four collections: 30 parent–offspring trios from Ibadan, 30 trios from Utah, 45 unrelated individuals from Beijing and 44 from Tokyo. Parent–offspring trios and the PHASE algorithm supplied haplotype estimation, rather than direct observation of every chromosome. In Figure 7, 36 SNPs resolve seven haplotypes in a chromosome 2 region in the Utah panel; seven tag SNPs capture the 36 SNPs at $r_{\mathrm{LD}}^2\ge 0.8$. The readout demonstrates reusable allelic correlation, not causal effects.

Controls required over 80% complete calls, at most one discordance among five duplicate genotypes, at most one Mendelian segregation error per trio panel, and a Hardy-Weinberg equilibrium test with $P>0.001$; blind reassessment checked performance. Minor allele frequencies of at least 0.05 were targeted, the collections were not population-representative, and inferred haplotypes retain phase uncertainty. These limits restrict extrapolation of linkage disequilibrium.^[ref]

Let the four haplotype frequencies sum to one and define $D=x_{AB}{x}_{ab}-x_{Ab}{x}_{aB}=x_{AB}-p_A{p}_B$. Recombination at rate $r$ removes the excess association: $\Delta x_{AB}=\Delta x_{ab}=-rD$ and $\Delta x_{Ab}=\Delta x_{aB}=rD$. Marginal allele frequencies are unchanged. Hence $D<0$ generates more $AB$, whereas $D>0$ removes it; a claim that recombination always increases this combination is false.

For two genetic loci with two alleles each, linkage equilibrium is independence of their states on sampled haplotypes. It concerns association between genetic loci; Hardy-Weinberg equilibrium concerns pairing of alleles into diploid genotypes.

For positive linkage disequilibrium $D$, take $D_{\max }=\min (p_A(1-p_B),(1-p_A)p_B)$; for negative $D$, take $D_{\max }=\min (p_A{p}_B,(1-p_A)(1-p_B))$ and retain its negative sign in $D^{\prime }$. Thus $|D^{\prime }|\le 1$. Its magnitude measures the fraction of the largest disequilibrium possible at the specified allele frequencies. High $|D^{\prime }|$ need not imply high allelic correlation, so it does not alone guarantee good marker prediction.

Genetic recombination erodes linkage disequilibrium by reshuffling alleles between haplotypes, while preserving marginal allele frequencies in the idealized model. Let $f_{AB,t}$ denote the haplotype frequency in the gamete pool of generation $t$, let $p,q$ denote its marginal allele frequencies, and let $r\in [0,1/2]$ be the constant recombination fraction between two autosomal genetic loci. Assume random mating within one population, nonoverlapping generations, fair Mendelian segregation, equal reciprocal gamete production, the same $r$ in both sexes, and a deterministic population large enough to neglect genetic drift. Exclude natural selection, mutation and gene flow. ThenHere $p$ and $q$ remain constant: recombining changes which alleles occur together, not their total numbers. The other haplotype frequencies change by $+r{D}_t$ for $Ab,aB$ and $-r{D}_t$ for $ab$. Consequently the joint probability distribution stays within its allowed bounds and approaches the product of its marginal distributions when $r>0$.

For the original positive example $p=0.60,q=0.40,D_0=0.12$, a recombination fraction $r=0.10$ gives $D_1=0.108$ and $D_{10}\approx 0.04184$. The negative example $D_0=-0.12$ has exactly the opposite sequence: genetic recombination reduces the magnitude without reversing the sign. With $r=0$, the initial linkage disequilibrium persists in this model; even independent assortment at $r=1/2$ only halves $D$ each generation. A population association does not vanish after one generation merely because the genetic loci are unlinked.

The geometric sequence gives a generation scale: for $r>0$, the continuous interpolation reaches $|D_0|/2$ at $t_{1/2}=\log (1/2)/\log (1-r)$. Actual generations are integers; the first at or below half is $\lceil t_{1/2}\rceil$. For small $r$, $t_{1/2}\approx (\log 2)/r$. Because the denominator of allelic correlation stays fixed, $r_{\mathrm{LD},t}^2=(1-r{)}^{2t}{r}_{\mathrm{LD},0}^2$; its decay is faster than the magnitude of $D_t$.

A short derivation uses random mating to assign frequency $2f_{AB}{f}_{ab}$ to the double heterozygote $AB/ab$ and $2f_{Ab}{f}_{aB}$ to $Ab/aB$. Relative to $r=0$, each loses or gains an $AB$ gamete fraction $r/2$, respectively. Their net contribution is $r(f_{Ab}{f}_{aB}-f_{AB}{f}_{ab})=-rD$. The other genotypes produce no net change in $f_{AB}$, proving the recurrence and, by iteration, its solution.

This temporal model does not identify elapsed generations from physical separation alone. The recombination fraction depends on genetic recombination, including recombination hotspots, while population stratification, natural selection, gene flow or genetic drift can create or maintain linkage disequilibrium. Repeated sampling across time would also require accounting for sample size and haplotype estimation. A cross-sectional plot of association against distance between genetic loci is therefore a different observation from the idealized trajectory plotted here.

For two segregating biallelic loci, encode the allele on a random population haplotype by indicator variables $X,Y$. Their covariance is the linkage disequilibrium coefficient $D$. The squared correlation is $r^2=D^2/[p_A(1-p_A)p_B(1-p_B)]$, between zero and one. It measures how well one marker predicts the other and is undefined if either marker is monomorphic.

// Target: biology.bigb