A finite-dimensional exponential family has common dominating measure and density
Here is the natural parameter of an exponential family, and normalization determines the cumulant function of an exponential family . A full regular family lets vary over an open natural parameter domain. A curved exponential family restricts to a lower-dimensional submanifold, while a minimal representation removes redundant affine combinations of the statistic.
Differentiating the normalization under regularity gives
Consequently is convex and its Hessian is the Fisher information matrix in natural coordinates. For an independent size- sample, is a sufficient statistic: the joint density factors into times , so the conditional law given loses its parameter dependence. An interior maximum-likelihood estimator solves
matching empirical and theoretical sufficient-statistic means. Under minimality the Hessian is positive definite, so the log-likelihood is strictly concave; existence of an interior maximizer still needs the observed statistic to be in the appropriate interior convex support.
For a full open natural family, the canonical sufficient statistic is a complete statistic: if throughout the open domain, exponential tilting about an interior point makes the Laplace transform of the integrable signed measure zero on a neighborhood. Transform uniqueness makes that measure zero, so almost surely. This requires integrability under the family. It is not an automatic property of curved restrictions. The sufficient-statistic reduction, explicit information and cumulant identities make these families especially useful for exact and higher-order inference. Normal distributions, Poisson distributions and gamma distributions supply familiar examples.
A transformation model is built differently: a group acts on the sample and parameter spaces, and its probability laws satisfy
In a transitive model all laws are transforms of a fixed reference law. The location family is generated by translations, with density . The positive-scale family has density , and the location-scale family combines them as
The Jacobian factor is essential. For independent observations its power is . The base density need not have moments, and a transformation model need not have fixed support.
An equivariant estimator respects the parameter action. With a well-defined equivariant fit valued in the transformation group, the normalized sample
is a maximal invariant. Equivariance gives ; if normalized samples agree, is obtained from by . In a transitive transformation model its law is parameter-free, so it is an ancillary statistic. For location and positive scale, centered standardized residuals provide this construction on nondegenerate samples. If the parameter action has a nontrivial stabilizer, normalization must retain its quotient action rather than pretending that a unique group element is determined.
This geometry motivates conditioning on an ancillary and using the P-star approximation for the fitted parameter. Ancillarity means parameter-free distribution; it does not by itself assert independence from the estimator. In a normal location model with known scale, the sample mean and centered residuals are independent by their joint Gaussian law and zero covariance. In an exponential scale model, the sample sum and its normalized proportions are independent, again an additional model property.
The two classes overlap but are distinct. A normal location-scale family is both an exponential family and a transformation model; an exponential mean family is both a one-dimensional natural exponential family and a scale model. A general Cauchy distribution location family is a transformation model without a fixed finite-dimensional regular exponential-family sufficient-statistic reduction. Exponential-family structure emphasizes sufficient statistics and natural-parameter convexity; transformation structure emphasizes equivariance, group orbits and ancillary reduction.
Consider a regular -parameter statistical model for which the data can locally be expressed as , where is the maximum-likelihood estimator and is a suitable exact or higher-order ancillary statistic. Write the log-likelihood as , and let
be its fitted observed information. The P-star approximation to the conditional sampling density of , given , is
The normalizing factor is independent of and is chosen so the density integrates to one over the feasible estimator coordinates. Its leading Gaussian value is , but exact normalization can change it. Both the fitted likelihood and the fitted information vary with ; the observed dataset is not held fixed while integrating over this sampling coordinate.
To see the structure, expand the likelihood difference for close to . Its leading term is minus one half of the information-weighted quadratic displacement, while supplies the local density scale. This recovers the first-order multivariate normal distribution of an efficient maximum-likelihood estimator. The formula is invariant under smooth one-to-one parameter changes: at a likelihood maximum the score vanishes, so the Hessian transforms as a quadratic form. Its square-root determinant supplies exactly the Jacobian determinant required for the estimator density to transform.
There is a direct derivation in a full regular exponential family. Let be the sufficient statistic for an independent sample, with likelihood
The maximum-likelihood equation is , and its derivative is . Applying a multivariate saddlepoint density approximation to gives
Changing variables from to multiplies by , producing the displayed P-star form. In more general models, ancillary conditioning supplies the relevant local sample coordinates; an ancillary is not an arbitrary extra statistic.
For an illustrative exact case, take an independent sample from an exponential distribution with mean , so . The fitted observed information is , and
Thus P-star has density shape . Normalizing gives
which is exactly the gamma distribution sampling density of the mean. Here sample ratios provide ancillary coordinates and are independent of the mean.
In suitable regular models the unnormalized leading construction has relative error of order , and higher-order ancillary constructions with normalization can attain relative order . Such accuracy needs the stated smoothness and ancillary hypotheses; it is not a universal consequence of asymptotic normality. The exact scale-model example is likewise a special property, not a general identity. The formula is useful for refined conditional inference, score distributions, and nuisance-likelihood adjustments, including modified profile likelihood. It describes a sampling density of the estimator, rather than a posterior density of the parameter.
An Edgeworth expansion refines a central normal approximation using higher cumulants. Let be independent and identically distributed, with mean , positive variance , and cumulants . Define
The logarithm of its characteristic function has the expansion
Expanding the exponential adds the squared cubic term . Fourier inversion turns into , where the probabilists' Hermite polynomials satisfy
Thus the density Edgeworth expansion through order is
Here , , and . Since , the corresponding cumulative distribution function expansion is
These remainder statements require adequate moments and smooth nonlattice conditions allowing transform inversion and error control, not merely a formal cumulant expansion. The density version needs the corresponding local smoothness. A symmetric distribution has , removing the first correction. Truncation can produce negative densities or nonmonotone distribution functions far from the center; it is not automatically a uniformly reliable relative tail approximation. A lattice distribution needs its lattice correction.
A Laplace approximation instead approximates an integral dominated by a sharp maximum. Suppose has a unique interior maximizer , with positive definite, and the integral is localized near that point. For smooth amplitude ,
Set . A Taylor expansion gives
Expanding and integrating the resulting Gaussian integral gives
Under sufficient smoothness and localization the relative correction is : odd terms of order integrate to zero over the limiting symmetric Gaussian region. Higher terms use derivatives of and Gaussian moments.
For a likelihood function with and a smooth positive prior density, this gives the integrated likelihood approximation
where is the observed information. Ratios of such integrals give posterior expectation by Laplace approximation; the same method is useful when integrating nuisance variables. Boundary maxima, singular Hessian matrices, nonlocalized tails and several dominant modes require altered formulas, with contributions summed when appropriate.
Both methods start with a quadratic Gaussian approximation and then retain higher-order terms. The Edgeworth series corrects a sampling distribution through its cumulants and Fourier inversion; Laplace's method corrects an integral through local derivatives at its dominant point. Neither should be confused with simply replacing a likelihood by a normalized posterior density.
Use the usual two mutually independent samples, and rank their pooled observations increasingly. Continuity gives no ties with probability one. Let be the pooled rank of and define the Wilcoxon rank sum test statistic
The last identity counts the ranks contributed by the other sample after subtracting the within- ranks.
When , the observations are stochastically smaller, so reject for small , equivalently small . For instance,
The inequality follows from and for continuous . A common-uniform quantile coupling also makes this direction transparent: , and is nondecreasing in each .
Under the common-distribution null hypothesis, the pooled observations are exchangeable. Every selection of ranks for the labels has probability . Thus the exact null law is
Choose a lower-tail critical value with probability at most the desired size, optionally randomizing at its boundary to attain the size exactly. This law is independent of the unknown common distribution. Common increasing transformations leave the pooled ranks of observations unchanged, so the statistic is a function of the maximal invariant from part (i). Although a rank sum is not itself maximal, it is an invariant test statistic. These facts justify the test within nonparametric statistics.
Sampling ranks without replacement gives
so with the same variance. For with , the normal approximation for a rank sum states
Equivalently replace by . The large-sample one-sided test rejects when this standardized statistic is below the lower normal distribution quantile. This is the two-sample rank-sum test, rather than the paired Wilcoxon signed-rank test.
Let a group act on the sample space. A statistic is a maximal invariant when it is invariant and separates group orbits:
Together these conditions say that exactly when and belong to the same orbit.
The basic result is that invariant tests factor through a maximal invariant. Specifically, a test function is invariant exactly when it can be written . To prove the nontrivial direction, define for any representative with . Two representatives are in the same orbit by maximality, and invariance gives them the same test value, so is well-defined. Conversely proves invariance. Measurability is taken on the quotient statistic space; for the finite rank spaces used below it is automatic.
If a null family is transitive under the group action, the null law of is parameter-free. Indeed, if is the law of when , then for every measurable set ,
Thus invariant procedures use exactly the information in , and its common null law calibrates tests without estimating the transformed nuisance distribution. This explains the role of maximal invariants in nonparametric statistics.
For simultaneous strictly increasing bijections of the real line, the labeled ranks of observations are a maximal invariant on samples without ties. Increasing maps preserve every comparison. Conversely, two such samples with the same ranks can be matched by a piecewise linear increasing bijection through their corresponding ordered observations, extended with positive-slope tails. This proves ranks as a maximal invariant under increasing transformations. Under independent sampling from any common continuous distribution, every labeled ordering is equally likely by exchangeability. Hence rank tests have distribution-free null calibration even without a global transitivity assertion for the entire class of continuous distributions.
For the unit-scale Laplace distribution, integrating separately over the two half-lines gives the moment-generating function
for ; the appropriate half-line integral diverges outside that interval. Therefore
Its derivatives needed for the saddlepoint density approximation are
Set . Solving and selecting the root in gives the stable formula
including at . The other quadratic root is outside the domain. At the correct saddlepoint,
Substitution gives the saddlepoint density of a Laplace sample mean:
It is symmetric in , as the underlying Laplace distribution requires. At zero its value is , consistent with the local normal approximation having variance . This is an approximation, not an exact finite- density or an automatically normalized one.
Let be the one-observation cumulant-generating function, finite on an open interval containing zero. At a value , find the real tilt solving
If the distribution is nondegenerate then in the interior, so the solution is unique whenever is in the range of . The leading saddlepoint density approximation to the sample mean is
To derive it, use exponential tilting:
Under this tilted distribution the mean is and the variance is , by differentiating its cumulant function . For , changing measure gives the exact density identity
At , is the tilted mean. A local normal approximation there gives . The Jacobian determinant for contributes the factor , yielding the boxed formula.
This approximation retains the exponential large-deviation factor, while the curvature is evaluated at the tilt appropriate to each . At the population mean, , so it reduces to the usual central normal approximation. Under suitable smooth nonlattice transform-inversion conditions, its relative error at fixed interior is ; existence of a few moments alone does not establish this density accuracy. Boundary values and absent exponential moments need separate treatment. The leading density need not integrate exactly to one, so it can be divided by its integral when a normalized approximation is desired.
For a coalition , its coalition excess at an allocation is , measuring its complaint. Sort all these excesses of a coalition in decreasing order. The nucleolus is the imputation whose sorted vector is smallest in lexicographic order: minimize the greatest complaint first, then the next among ties, and continue. Including the empty and grand coalitions merely inserts two fixed zero entries and does not change the minimizing allocation.
For every efficient allocation, the excesses of a coalition of and sum to zero:
The largest proper-coalition coalition excess is therefore at least zero. Since the core of a cooperative game from part (b) is nonempty, this first-stage minimum is exactly zero and its minimizers are precisely that core of a cooperative game. Restrict to , . The six proper-coalition excesses of a coalition are
The two zero excesses of a coalition are fixed. Among the remaining excesses of a coalition, , so the next largest complaint is . It is minimized by equalizing those two affine expressions:
This minimizer is unique, so later lexicographic stages cannot change it. Thus
Its sorted proper-coalition coalition excess vector is . The calculation both identifies the nucleolus and proves uniqueness for this game without replacing the requested computation by the general uniqueness theorem.
The core of a cooperative game is the set of efficient allocations for which every coalition receives at least its own value: for every . The singleton constraints give individual rationality, and the remaining proper-coalition inequalities are
Efficiency turns the second into ; together with this forces . Put and . The remaining constraints give , , and from . The first pair constraint adds only . Hence
Each point in this segment satisfies all singleton, pair, and grand-coalition constraints, so the description is sufficient as well as necessary. Its endpoints are and .
For a transferable utility game, an imputation is an efficient and individually rational payoff allocation:
Here it is
Writing , , identifies it with a scaled standard simplex . Its three vertices are , and .
The objective function is nonnegative. We now construct a feasible zero-objective point, distinguishing the input problem's possible states.
If the primal is feasible and bounded below, strong duality and attainment for a linear program give , and with
Set , , , , , and
The first three embedding equations hold. Substituting their equalities into the fourth gives precisely .
If the primal is infeasible, the Farkas alternative supplies with and . Take , , , and . These satisfy the same equations. If the primal is feasible and unbounded below, it has a direction in the recession cone with and . Set , , and . This is again a feasible nonzero embedding vector. Normalize the constructed vector by the sum of its nonnegative coordinates. In every case,
The facts about optimal pairs and recession directions follow from the polyhedral alternatives of linear programming duality; they do not require knowing their values or witnesses to write the transformed coefficients.
To recover information from the transformed optimum, multiply the first three original embedding equations by , , and , respectively, and add. Their cross terms cancel; the fourth equation gives
At , nonnegativity forces and . The equations also imply
Thus any normalized zero-objective point has : if , all these coordinates vanish, and then , contradicting normalization.
If , division by gives a primal-feasible point and dual-feasible point with equal objective function values; hence they are optimal. If , then , , , and
Therefore either certifies primal infeasibility, or is an improving recession direction certifying dual infeasibility and unboundedness whenever the primal is feasible. If only the latter certificate is obtained and primal feasibility is unknown, apply the same construction to the feasibility problem with objective function ; it gives either a feasible point or a direct primal infeasibility certificate.
A degenerate optimal point can have and convey neither conclusion. Choose an optimum in the relative interior of the optimal face of a linear program. The explicit existence argument above gives a point of this face with . A nonnegative coordinate which is positive somewhere on a face of a convex set is positive throughout its relative interior, so the chosen point has and can be decoded. The assertion follows because a small relative-interior displacement in the direction opposite a positive-coordinate point would otherwise make that coordinate negative. Equivalently one may obtain a maximum-support optimum; choosing an arbitrary degenerate boundary optimum is insufficient.
This reduction is useful because an interior-point method now has a known strictly feasible start, a compact standard simplex slice, and a known target objective function. Projective rescalings can recenter positive iterates, and the transformed problem supplies optimality or infeasibility information without a separate guessed optimum. Polynomial bit-complexity claims need rational data with a finite encoding length. The printed bound on arbitrary real coefficients by itself does not bound denominators or the size of nonzero determinants; it is not a substitute for that encoding assumption.
In the unnormalized embedding, take
Then the first equation is , the second is , and the third is . The fourth becomes
The multiplier elimination preserves this point because and hence . Every coordinate of is one. Dividing by therefore gives
which satisfies , , and strict positivity. Its existence is independent of feasibility of the original input problem.
For this fraction, and , so their product is . The signed defining relation is
The complete factorial contrast alias classes are
The five main effects and occupy six distinct alias classes, and none aliases with the intercept. Every undesired two-factor term in these classes is assumed negligible, and all terms of order at least three are also assumed negligible. Thus design (b) is suitable under the stated interaction assumptions. Its intercept and six requested columns are mutually orthogonal, with , so each coded regression coefficient is . The corresponding high-minus-low main effect is .
There is one residual degree of freedom, but no replicated setting and therefore no separate pure error estimate. Suitability here means estimability under the stated model; it does not protect estimates from active interactions that the assumptions exclude.
On these eight runs, and . The two independent defining words therefore give the full signed defining relation
Multiplying each desired factorial contrast by its four words gives all aliases. In the following equalities a minus sign means opposite contrast columns:
Thus two main effects, and , are inseparable even when every unwanted interaction vanishes. In addition, the important interaction aliases with the main effect . Design (a) is unsuitable. The seven-column design matrix for the intercept, the five main effects and has rank five, confirming the two independent estimability failures.
Fix one treatment. Each of its blocks supplies companion treatments, giving incidences. Alternatively, each of the other treatments accompanies it in exactly blocks. Thus
Count incidences of a treatment with a block containing it. Summing over blocks gives , while summing over treatments gives . Hence the balanced incomplete block design satisfies
Multivariate analysis of variance tests differences between group mean vectors when each experimental unit has several correlated responses. For independent -dimensional observations in groups, use a common positive-definite covariance matrix and the multivariate normal model . The null hypothesis is equality of all means.
Write , group means , and the size-weighted grand mean . The within-group and between-group scatter matrices are
Expanding deviations about the group means yields total scatter , with zero cross terms. Under the null, orthogonal normal projections give independent Wishart distributions and .
With nonsingular , the Wilks lambda statistic is
where the nonnegative are eigenvalues of . Maximizing the common covariance under the two mean models gives the likelihood ratio , so small indicates large group separation relative to within-group spread. For fixed and large group sizes, the Wilks theorem gives under the null; a finite-sample reference distribution can also be used.
The lower-right sketch shows two elongated groups whose centroid difference lies nearly across, rather than along, their long axes. Their correlation makes this multivariate separation much clearer than either marginal mean shift alone. Nonsingular linear rescaling changes and by congruence, leaving the determinant ratio unchanged. MANOVA measures mean separation against the full within-group covariance geometry. Its interpretation requires independent observations and an adequate common-covariance model; separate coordinatewise tests discard that geometry and raise multiple-testing issues.
Agglomerative hierarchical clustering starts with each observation as a singleton cluster in cluster analysis, repeatedly joins the closest pair of current clusters, and continues until one remains. The input is a dissimilarity matrix, so choosing variable scales and a scientifically sensible dissimilarity is part of the analysis.
Different linkages define closeness differently:
Single-linkage clustering can connect elongated chains through nearest neighbors; complete-linkage clustering emphasizes compact clusters; average-linkage clustering averages all cross-pair distances. For Euclidean observations, Ward minimum-variance clustering instead chooses the smallest increase in within-cluster squared error,
This follows by expanding squared deviations around the merged mean and using the zero sums of within-cluster residuals.
A dendrogram plots merge heights. In the lower-left sketch, two close pairs first merge at small heights, then join across a much larger separation; a horizontal cut between these heights yields two groups. The hierarchy gives nested partitions, not a uniquely established number of populations. Examine sensitivity to scaling, linkage and ties; once an agglomerative merge is made, the method does not undo it. The absence of a probability model also means that a visually attractive dendrogram is not itself a significance test.

Pinned article: Introduction to the OurBigBook Project

Welcome to the OurBigBook Project! Our goal is to create the perfect publishing platform for STEM subjects, and get university-level students to write the best free STEM tutorials ever.
Everyone is welcome to create an account and play with the site: ourbigbook.com/go/register. We belive that students themselves can write amazing tutorials, but teachers are welcome too. You can write about anything you want, it doesn't have to be STEM or even educational. Silly test content is very welcome and you won't be penalized in any way. Just keep it legal!
We have two killer features:
  1. topics: topics group articles by different users with the same title, e.g. here is the topic for the "Fundamental Theorem of Calculus" ourbigbook.com/go/topic/fundamental-theorem-of-calculus
    Articles of different users are sorted by upvote within each article page. This feature is a bit like:
    • a Wikipedia where each user can have their own version of each article
    • a Q&A website like Stack Overflow, where multiple people can give their views on a given topic, and the best ones are sorted by upvote. Except you don't need to wait for someone to ask first, and any topic goes, no matter how narrow or broad
    This feature makes it possible for readers to find better explanations of any topic created by other writers. And it allows writers to create an explanation in a place that readers might actually find it.
    Figure 1.
    Screenshot of the "Derivative" topic page
    . View it live at: ourbigbook.com/go/topic/derivative
  2. local editing: you can store all your personal knowledge base content locally in a plaintext markup format that can be edited locally and published either:
    This way you can be sure that even if OurBigBook.com were to go down one day (which we have no plans to do as it is quite cheap to host!), your content will still be perfectly readable as a static site.
    Figure 5. . You can also edit articles on the Web editor without installing anything locally.
    Video 3.
    Edit locally and publish demo
    . Source. This shows editing OurBigBook Markup and publishing it using the Visual Studio Code extension.
  3. https://raw.githubusercontent.com/ourbigbook/ourbigbook-media/master/feature/x/hilbert-space-arrow.png
  4. Infinitely deep tables of contents:
    Figure 6.
    Dynamic article tree with infinitely deep table of contents
    .
    Descendant pages can also show up as toplevel e.g.: ourbigbook.com/cirosantilli/chordate-subclade
All our software is open source and hosted at: github.com/ourbigbook/ourbigbook
Further documentation can be found at: docs.ourbigbook.com
Feel free to reach our to us for any help or suggestions: docs.ourbigbook.com/#contact