A non-degenerate probability distribution is not concentrated at a single point. In extreme value theory, a non-degenerate distribution function is max-stable if, for each integer , constants satisfy for every . This says that a normalized sample maximum has the same law as one observation. A distribution function belongs to the maximum domain of attraction of a non-degenerate if there are such that
at every continuity point of . Equivalently, has convergence in distribution to .
The extremal types theorem says that any such non-degenerate limit is max-stable and, up to a positive affine change of variable, is exactly one of the following:
These are respectively the Gumbel distribution, Fréchet distribution, and negative Weibull distribution. The theorem is also known as the Fisher–Tippett–Gnedenko theorem.
Useful sufficient conditions can be expressed using the survival function and the right endpoint of a distribution . An infinite endpoint with for every gives the Fréchet distribution domain. A finite endpoint with as gives the negative Weibull distribution domain. For the Gumbel distribution, it suffices that a positive Gumbel auxiliary function gives for every real as . These are regular variation and exponential tail-ratio conditions; they need no proof here.
In case (i), on , so the finite-endpoint ratio is exactly . Therefore the domain is negative Weibull with shape . With and ,
and the limit is one for .
In case (ii), has exponential tail ratio with the constant Gumbel auxiliary function . Therefore the domain is Gumbel. Choosing and gives
In case (iii), has regular variation of index , so the domain is Fréchet with shape . The convenient choices and give
with limit zero for .
Finally, define the empirical distribution function . On a sample with positive threshold and exactly observations strictly above , its survival function has . Using the Tonelli theorem to integrate the finite nonnegative sum gives
This is the Hill estimator as an empirical plug-in version of the tail-integral limit. It does not prove consistency for fixed ; that is a separate issue.
There is a finite-sample qualification because the question permits ties and does not assume positive observations. The formula requires and . If because the threshold is tied, direct substitution gives
where terms equal to the threshold contribute zero. This is generally times the displayed Hill estimator, rather than the same estimator. If , the empirical plug-in denominator is zero. For a continuous probability distribution the no-tie condition holds almost surely; for a Fréchet distribution domain and fixed , the threshold is positive with probability tending to one. Thus the stated plug-in identity holds at a positive untied threshold, and the literal claim for all samples from an arbitrary distribution function needs this qualification.
For fixed-design nonparametric regression, let and . The degree- local polynomial estimator takes the fitted intercept , where
Use a nonnegative regression kernel and an invertible local polynomial Gram matrix to obtain a unique fit. For , solving the single weighted least squares normal equation gives the local constant estimator:
At the interior point , use the standard unit-integral regression kernel convention . Otherwise the leading variance below is ; unlike a kernel density estimator, the local constant estimator itself is unchanged by rescaling .
The local constant estimator is a linear estimator in nonparametric regression with weights . The independent unit-variance errors imply
To control the bias of an estimator, differentiability at this one point is enough. Write
Symmetry makes , and the supplied moment approximation gives . Therefore the linear contribution to the bias of an estimator is . The remaining contribution has absolute value at most , because only receive positive weight. This is the interior first-order bias cancellation for local constant regression, and gives squared bias of an estimator . Combining with the variance proves
For fixed , take . It satisfies both asymptotic smoothing bandwidth conditions, and the displayed remainder, multiplied by , tends to zero for this fixed . Thus
Letting proves the optimized scaled pointwise error tends to zero. The order of these limits matters: no remainder uniform in has been assumed.
For the lower bound over the whole function class, the absence of a uniform bound on derivatives permits a particularly direct argument. It also avoids assuming a normal distribution for errors when the model only supplies their first two moments. Fix and . Choose a smooth bump function on the interval, with , whose support contains no design point except possibly . Such a bump exists because the design is finite; at an endpoint use the restriction of a smooth bump on the real line. Every , , belongs to the differentiability class.
If is not a design point, all observation means are zero for every . The Le Cam two-point lemma applied to has total variation distance zero and gives a lower bound tending to infinity with . In fact, the supremum of the mean squared error is infinite at every such .
If is a design point, only its response depends on . The resulting location family is the single observation ; all remaining observations are independent of and give no extra information. The single-observation location minimax bound is
Here is a proof covering discrete as well as continuous errors. Give the prior distribution , independently of the noise. Put and . Consider an easier experiment in which an oracle also reveals whether , and reveals itself on the complementary event. On the event and , every possible truncated noise value places within . The flat prior distribution then makes the Bayesian posterior noise law exactly its truncated original law. The smallest conditional squared-error loss is its conditional variance , attained by the posterior mean.
The probability of that event is at least for , by restricting further to . Thus every rule's supremum risk function is at least the original Bayes risk, which is at least the easier experiment's Bayes risk, and therefore at least . Let and then . Finite second moment and zero mean give . The rule has constant mean squared error one, proving the equality.
Consequently, at every , the unrestricted differentiability class satisfies the stronger conclusion
Thus works for every . There is no conflict with the preceding fixed-function limit: the worst functions, including arbitrarily narrow bumps, can vary with . This is the pointwise versus uniform risk distinction. A smoothness ball with a common derivative bound would pose a different minimax risk problem.

Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 5. . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact