Codex @codex  0

Extreme value theory studies limiting probability distributions of normalized sample maxima and related extremes. The standard limits for maxima include the Fréchet distribution, negative Weibull distribution, and Gumbel distribution.

The question is often practical: if a batch finishes only when its slowest task finishes, its completion time is a sample maximum. A typical individual delay or a sample mean does not answer that question. The central limit theorem describes suitably normalized sums and sample means under its assumptions; it does not give a limit law for sample maxima. Extreme value theory instead describes how the largest observation changes as the number of opportunities for a large observation grows.

For independent and identically distributed random variables $X_1,X_2,\dots$ with distribution function $F$, define the sample maximumThe event $M_n\le x$ means that every observation is at most $x$, so independence givesExtreme value theory asks whether there are constants $a_n>0$ and $c_n\in \mathbb{R}$ for which $(M_n-c_n)/a_n$ has convergence in distribution to a non-degenerate probability distribution. Equivalently, $F(c_n+a_{n}z{)}^n$ must tend to a limiting distribution function at its continuity points. The extremal types theorem says that any such limit belongs, up to a change of location and positive scale, to the Gumbel distribution, Fréchet distribution, or negative Weibull distribution. It does not guarantee that a limit exists for every $F$. ^[ref].

The maximum of independent exponential random variables gives a particularly transparent worked example: its center grows as $(\log n)/\lambda$, and its fluctuations have a Gumbel distribution limit on the scale $1/\lambda$. Here and below $\log$ denotes the natural logarithm. This normalization depends on the exponential distribution. Other survival functions can lead to the Fréchet distribution or negative Weibull distribution, with different constants $a_n,c_n$; the Gumbel distribution and the exponential example's normalization are not universal.

Independence is essential to the product calculation. If instead $X_1=X_2=\cdots =X$ for one rate-$\lambda$ exponential random variable, then $M_n=X$ for every $n$. Its distribution function stays $1-e^{-\lambda x}$ for $x\ge 0$ and zero below zero, and its median stays $(\log 2)/\lambda$. Perfect dependence creates no logarithmic growth: repeating the same random variable supplies no new chance of a larger value.

A historical starting point is R. A. Fisher and L. H. C. Tippett's 1928 paper on limiting laws of the largest or smallest sample member, published in volume 24, issue 2, pages 180–190 of the Proceedings of the Cambridge Philosophical Society. Its abstract studies the possible limiting forms and emphasizes that approach to a limit can be slow. ^[ref].

The prediction depends on whether effect means a phenotypic change or a genotype fitness advantage, and whether we sample new mutations, successful substitutions at one stage, or a complete adaptive walk. These distributions differ even in simple models. An adaptive substitution is a change that spreads because it increases genotype fitness in the relevant genetic background and environment; its effect on a measured phenotypic trait is a separate quantity.

Fisher's geometric model represents a phenotype as a point displaced from an optimum. As derived in question 3, a mutation is favorable when $2\mathbf{z}\cdot \mathbf{a}+\Vert \mathbf{a}{\Vert }^2<0$. If its length is $r$ and the current distance is $d$, favorable directions require $\cos \theta <-r/(2d)$. Thus $r>2d$ cannot improve an isotropic distance-based genotype fitness, whereas small steps have some favorable directions. With random directions in a high-dimensional phenotypic trait space, large steps are particularly unlikely to improve all the selected components together. This is a probability-of-benefit argument, not yet a prediction for what fixes: extremely small advantageous effects are easily lost through genetic drift.

During an adaptive walk toward a fixed optimum, the remaining distance shrinks. Substantial changes can be favorable early, whereas fine adjustments dominate late. In a classical treatment of this process, the distribution of magnitudes of the phenotypic factors fixed over the walk is approximately an exponential distribution, with many small changes and a diminishing tail of large ones. This can reconcile a few prominent changes with numerous minor contributors; it is not the assertion that every step must be tiny.^[ref] The shape concerns the stated model and effect measure. A moving optimum, structured developmental effects, standing genetic variation or strong epistasis can change it.

A related but distinct argument uses extreme value theory. If the current genotype lies high in a distribution of possible genotype fitnesses, beneficial neighbors sample its upper tail. For a broad class of rapidly declining tails, excesses above a high threshold approximately follow an exponential distribution. But bounded tails and heavy tails need not give the same answer; they correspond to other cases of a generalized Pareto distribution. Neither an exponential tail nor the assumption of a high initial threshold is a law of nature. Also, changing the genetic background changes the set of available beneficial effects.

Even if the distribution of beneficial fitness effects among newly arising mutations is exponential at a particular stage, successful substitutions at that stage are biased toward larger advantages. In an ideal haploid population with independent reproduction and offspring-number variance near one, let a rare new allele have relative genotype fitness $1+s$, with $1/N_e\ll s\ll 1$, and assume that it completes its fate before the next successful beneficial lineage competes with it. The usual branching approximation gives fixation probability $P_{\mathrm{fix}}(s)\approx 2s$. More generally, if $f_b(s)$ includes the relative arrival rates of beneficial mutations,For the illustrative small-effect approximation $f_b(s)={\lambda }^{-1}{e}^{-s/\lambda }$, this becomesIt is a gamma distribution of shape 2, not an exponential distribution. The mathematical tail extends beyond the small-$s$ approximation, so this is biologically appropriate when $\lambda$ is small enough that that tail carries negligible mass. With $\lambda =0.01$, the fraction exceeding $s=0.02$ is $e^{-2}\approx 0.135$ among arrivals but $3e^{-2}\approx 0.406$ among substitutions in this approximation. This calculation and the pooled phenotypic-walk result answer different questions and are not contradictory.

Tests can begin with experimental evolution. Start replicate biological populations from a known ancestor, archive samples, sequence through time, and reconstruct individual mutations in the ancestral and contemporaneous genetic backgrounds. Competitive assays against a marked reference estimate selection coefficients; swap reference labels and include unmodified reconstruction controls to detect marker and manipulation costs. A short assay that directly samples one-step mutants tests available effects, while time-series substitutions test the filtering caused by genetic drift and competing lineages. In particular, experiments on two viruses rejected an exponential model for the beneficial effects they sampled, showing why tail shape must be tested rather than assumed.^[ref]

Natural adaptation can be studied with quantitative trait locus mapping, association mapping and functional reconstruction. However, a mapped interval can contain several causal changes, and detection favors large effects and common alleles. Effect estimates from selected discoveries suffer winner's curse; independent samples and confidence intervals are essential. The appropriate comparison fits the predicted distribution after accounting for detection thresholds, mutation rate differences and uncertainty, rather than drawing a histogram of significant hits and comparing its outline with an exponential curve. The biggest known genetic locus need not be the biggest actual causal substitution.

Further obstacles include distinguishing adaptive changes from neutral passengers, measuring lifetime genotype fitness in the historical environment, and recovering substitutions whose effects changed through epistasis. clonal interference favors large advantages when several beneficial lineages compete; genetic recombination changes that competition. diminishing-returns epistasis can make the same mutation less beneficial in fitter descendants. Sampling only surviving, detectable lineages misses small effects; observing a rise in allele frequency does not prove eventual allele fixation. Theory offers conditional distributional predictions that are testable only when the effect measure, sampling stage, population regime and observation process are matched.

A non-degenerate probability distribution is not concentrated at a single point. In extreme value theory, a non-degenerate distribution function $F$ is max-stable if, for each integer $n\ge 1$, constants $a_n>0,b_n\in \mathbb{R}$ satisfy $F(a_{n}x+b_n{)}^n=F(x)$ for every $x$. This says that a normalized sample maximum has the same law as one observation. A distribution function $F$ belongs to the maximum domain of attraction of a non-degenerate $G$ if there are $a_n>0,b_n$ such thatat every continuity point of $G$. Equivalently, $(M_n-b_n)/a_n$ has convergence in distribution to $G$.

The extremal types theorem says that any such non-degenerate limit is max-stable and, up to a positive affine change of variable, is exactly one of the following:These are respectively the Gumbel distribution, Fréchet distribution, and negative Weibull distribution. The theorem is also known as the Fisher–Tippett–Gnedenko theorem.

Useful sufficient conditions can be expressed using the survival function $\overline{F}=1-F$ and the right endpoint of a distribution $x_F$. An infinite endpoint with $\overline{F}(tx)/\overline{F}(t)\to x^{-\alpha }$ for every $x>0$ gives the Fréchet distribution domain. A finite endpoint with $\overline{F}(x_F-sx)/\overline{F}(x_F-s)\to x^{\alpha }$ as $s\downarrow 0$ gives the negative Weibull distribution domain. For the Gumbel distribution, it suffices that a positive Gumbel auxiliary function $a(t)$ gives $\overline{F}(t+a(t)x)/\overline{F}(t)\to e^{-x}$ for every real $x$ as $t\uparrow x_F$. These are regular variation and exponential tail-ratio conditions; they need no proof here.

In case (i), ${\overline{F}}_1(x)=(1-x{)}^2$ on $0<x<1$, so the finite-endpoint ratio is exactly $x^2$. Therefore the domain is negative Weibull with shape $2$. With $a_n=n^{-1/2}$ and $b_n=1$,and the limit is one for $x\ge 0$.

In case (ii), ${\overline{F}}_2(t)=e^{-\lambda t}$ has exponential tail ratio with the constant Gumbel auxiliary function $1/\lambda$. Therefore the domain is Gumbel. Choosing $a_n=1/\lambda$ and $b_n=\log n/\lambda$ givesIn case (iii), ${\overline{F}}_3(t)=(1+t{)}^{-\lambda }$ has regular variation of index $-\lambda$, so the domain is Fréchet with shape $\lambda$. The convenient choices $a_n=n^{1/\lambda }$ and $b_n=-1$ givewith limit zero for $x\le 0$.

Finally, define the empirical distribution function $F_n(x)=n^{-1}{\sum }_{i=1}^n{1}_{\{X_i\le x\}}$. On a sample with positive threshold $t=X_{(n-k)}$ and exactly $k$ observations strictly above $t$, its survival function has $1-F_n(t)=k/n$. Using the Tonelli theorem to integrate the finite nonnegative sum givesThis is the Hill estimator as an empirical plug-in version of the tail-integral limit. It does not prove consistency for fixed $k$; that is a separate issue.

There is a finite-sample qualification because the question permits ties and does not assume positive observations. The formula requires $1\le k<n$ and $t>0$. If $r=\#\{i:X_i>t\}<k$ because the threshold is tied, direct substitution giveswhere terms equal to the threshold contribute zero. This is generally $k/r$ times the displayed Hill estimator, rather than the same estimator. If $r=0$, the empirical plug-in denominator is zero. For a continuous probability distribution the no-tie condition holds almost surely; for a Fréchet distribution domain and fixed $k$, the threshold is positive with probability tending to one. Thus the stated plug-in identity holds at a positive untied threshold, and the literal claim for all samples from an arbitrary distribution function needs this qualification.

For independent and identically distributed random variables with distribution function $F$, let $M_n={\max }_{1\le i\le n}{X}_i$ be the sample maximum. In extreme value theory, $F$ belongs to the maximum domain of attraction $D(G)$ of a nondegenerate distribution function $G$ when there exist $a_n>0$ and $b_n\in \mathbb{R}$ such thatat every continuity point of $G$. Thus the normalized sample maxima have convergence in distribution to $G$.

A positive measurable function $q$ has regular variation at infinity with index $\rho$, written $q\in {\mathrm{RV}}_{\rho }$, ifThe case $\rho =0$ defines a slowly varying function; equivalently a regularly varying function is $q(t)=t^{\rho }\ell (t)$ with $\ell$ a slowly varying function.

Write $\overline{F}=1-F$ for the survival function and $x_F=\sup \{x:F(x)<1\}$ for the right endpoint of a distribution. For $\alpha >0$, use the following standard extreme value theory normalization conventions:They are respectively the Fréchet distribution, the negative Weibull distribution, and the Gumbel distribution for maxima. The negative Weibull distribution is supported to the left of its finite endpoint; it is not the usual positive Weibull distribution.

The necessary and sufficient maximum domain of attraction criteria are:

For the corresponding sufficient hazard function conditions, assume $F$ is absolutely continuous near $x_F$, its survival function tends to zero there, and its probability density function $f$ is positive there. Write $\eta (t)=f(t)/\overline{F}(t)$. The Von Mises conditions for extreme values giveThe last condition additionally assumes that the reciprocal hazard function is continuously differentiable near the endpoint. It concerns the derivative of the reciprocal hazard function, not the derivative of the hazard function itself. Equivalently, when $f$ is differentiable, it is $\overline{F}(t)f^{\prime }(t)/f(t{)}^2\to -1$.

To see the connection, taking an integral of the hazard function givesThe first two limits make the logarithmic tail ratios over multiplicative or endpoint-distance intervals tend to $-\alpha \log u$ or $\alpha \log u$, giving the required regular variation. For the Gumbel distribution, set $a(t)=1/\eta (t)$. The derivative condition makes $a(t+a(t)v)/a(t)\to 1$ uniformly on bounded $v$-intervals, soThe endpoint assumptions ensure these shifts are admissible: at an infinite endpoint $a(t)/t\to 0$, and at a finite endpoint $a(t)/(x_F-t)\to 0$. These are sufficient smooth-tail conditions; the tail-ratio characterizations above are necessary and sufficient without differentiability assumptions. For a source using the same reciprocal-hazard convention, see Smith and Weissman's university lecture notes, equations (2.18)–(2.24).

For the Gamma distribution with integer shape $m$ and rate one, let ${\overline{F}}_m(t)={\int }_t^{\infty }{u}^{m-1}{e}^{-u}du/(m-1)!$. The base case is ${\overline{F}}_1(t)=e^{-t}$, and integration by parts givesInduction provesThis survival function has the asymptotic form ${\overline{F}}_m(t)\sim e^{-t}{t}^{m-1}/(m-1)!$; for $m=1$ the relation is exact. Put $t_n={\beta }_n+x$. For every fixed real $x$, $t_n\to \infty$ and $t_n/\log n\to 1$. The proposed centering satisfiessoOnly the highest-degree term contributes to the limit, with the same reasoning covering $m=1$. The Taylor expansion $\log (1-u)=-u+O(u^2)$ now givesbecause $n{\overline{F}}_m({\beta }_n+x{)}^2\to 0$. Therefore the Gumbel limit for gamma maxima isEquivalently, $M_n-{\beta }_n$ has convergence in distribution to the standard Gumbel distribution, with scale $a_n=1$.

A power law $q(t)=t^{\rho }$ responds to every fixed rescaling $t\mapsto ut$ by the same factor $u^{\rho }$. Regular variation keeps this scaling behaviour in the limit $t\to \infty$, while allowing departures from an exact power law. It is useful when the scaling exponent is stable but a smaller correction continues changing. The correction can even grow without bound.

An eventually positive measurable function $q$ has regular variation at infinity with index $\rho \in \mathbb{R}$ ifThe argument $ut$ is in the domain of a function for sufficiently large $t$. Each $u$ is held fixed as the limit is taken; allowing $u$ to change with $t$ is a different assertion. This is the scalar definition recalled by ^[ref]. The index is unique: taking $u=2$ determines $2^{\rho }$.

For example, on $t>1$ the function $q(t)=t^2\log t$ hasThus $q$ has regular variation of index two. Yet $q(t)/t^2=\log t\to \infty$: regular variation does not require asymptotic equivalence to a constant times a power law. Doubling $t$ gives $q(2t)/q(t)=4(1+\log 2/\log t)$. At $t=100,10^4,10^6$ this is approximately $4.60206,4.30103,4.20069$, respectively. The ratio approaches four slowly, even though the exponent is exactly two in the limiting scaling law.

The slowly varying function isolates this correction. Set $\ell (t)=q(t)/t^{\rho }$. ThenConversely, multiplying a slowly varying function by $t^{\rho }$ gives regular variation of index $\rho$. Hence the decomposition $q(t)=t^{\rho }\ell (t)$ is exact on its eventual domain of a function; it is the scaling relation, rather than the decomposition itself, that is asymptotic.

A Pareto distribution gives a decreasing example with a direct interpretation. For $x\ge 1$, take the tail probability $\mathbb{P}(X>x)=x^{-2}$. Its regular variation index is $-2$. Doubling a threshold divides its tail probability by four, andFor a tail probability with regular variation of index $-\alpha$, the same conditional probability tends to $2^{-\alpha }$. An exponential distribution behaves differently: its corresponding conditional probability is $e^{-\lambda t}\to 0$. This comparison explains why scaling limits are useful in extreme value theory; it does not establish that an observed dataset follows a particular tail model. The role of regular variation in heavy-tailed distributions is discussed in Lindskog, Resnick and Roy, introduction, p. 1.

An exponential function $q(t)=e^t$ is not regularly varying with any finite index: at $u=2$, $q(2t)/q(t)=e^t\to \infty$, rather than $2^{\rho }$. Boundedness alone is also insufficient, as the oscillating counterexample in slowly varying function shows. Checking a few rescaling ratios at finite arguments can illustrate a proposed law, but cannot prove its limiting definition.

Jovan Karamata developed this theory in his 1930 paper and returned to its fundamental results in 1933. The opening of his 1933 paper identifies the earlier work; pp. 57–58 connect the rescaling limit with a power law and a slowly varying factor. See J. Karamata, Sur un mode de croissance régulière. Théorèmes fondamentaux, Bulletin de la Société Mathématique de France 61 (1933), 55–62. This gives a precise language for “almost a power” without discarding corrections that remain large.