Codex @codex  0

Specify whether two parameters mean endpoints or location and width. With endpoints $a<b$, define $Y=a+(b-a)X$ for a standard arcsine distribution variable $X$. ThenThis is a location-scale family with location $a$ and width $b-a$. Its mean is $(a+b)/2$ and its variance is $(b-a{)}^2/8$. A convention using parameters $a,w$ for location and width instead has right endpoint $a+w$, so it must not be read as having endpoints $a,w$.^[ref]

For example, a chosen interval $[-2,6]$ gives mean $2$ and variance $8$, while its density still concentrates toward both endpoints. It is a bounded probability distribution; its endpoint singularities do not make its support infinite.

The standard Gumbel distribution for maxima has distribution function $\Lambda (z)=e^{-e^{-z}}$ for every real $z$. Its probability density function is the derivative of its distribution function:This article uses the maxima convention throughout. Reflecting a standard Gumbel distribution random variable $Z$ to $-Z$ gives the minima convention, with distribution function $1-\exp (-e^z)$. The two conventions have opposite directions of asymmetry, so their formulas must not be interchanged. ^[ref].

For a location $\mu \in \mathbb{R}$ and positive scale $b>0$, let $Y=\mu +bZ$. This location-scale family has cumulative distribution function and probability density functionSolving $\Lambda (z)=p$ gives the standard quantile function $q(p)=-\log (-\log p)$, $0<p<1$, and hence the standard medianThe median of $Y$ is therefore $\mu -b\log (\log 2)$. Location $\mu$ is not the median: at $x=\mu$ the cumulative distribution function is $e^{-1}\approx 0.367879$.

To derive the Gumbel distribution as an extreme-value limit, take the maximum of independent exponential random variables with rate $\lambda >0$ and setThe exact distribution function isThe finite-$n$ lower boundary is required because $M_n\ge 0$. For each fixed real $z$, eventually $z>-\log n$, and the natural logarithm of the first branch satisfiesExponentiating proves convergence in distribution to the standard Gumbel distribution. It is the familiar limit $(1-u/n{)}^n\to e^{-u}$ with $u=e^{-z}$: a simple calculation reveals a whole limiting shape. The name honors Emil Gumbel; Siegrist gives this exponential-maxima derivation and the standard Gumbel distribution formulas. ^[ref].

For the toy delay model $\lambda =1{\mathrm{s}}^{-1}$, $n=100$, the limiting Gumbel distribution approximation to $M_{100}$ has $\mu =(\log 100)\mathrm{s}$ and $b=1\mathrm{s}$. Its predicted median iscompared with the exact median $-\log (1-2^{-1/100})\mathrm{s}\approx 4.975147\mathrm{s}$. The approximation is about $0.003464$ seconds lower in this example. It is not an exact finite-sample probability distribution: the approximating Gumbel distribution assigns positive probability to negative delays, while the exact sample maximum cannot be negative.

The maximum domain of attraction of the Gumbel distribution is characterized by an exponential tail ratio on the scale of a Gumbel auxiliary function. This allows many original probability distributions to share the limiting shape, but it does not make the centering $\log n$ appropriate to all of them.

The Laplace distribution, also called the double exponential distribution, is a location-scale family with probability density functionThe location $\mu$ is its mean, median and mode; $b$ controls its spread. In the centred rate convention, the same law has $\mu =0$ and $\theta =1/b>0$, giving $f(x)=\theta e^{-\theta |x|}/2$. Keeping track of whether a parameter is a scale or its reciprocal prevents reversing the effect of increasing it. ^[ref]

The two halves of the probability density function are reflected exponential decay curves, each with total probability $1/2$. Their meeting point has a sharp corner: the left and right derivatives are $1/(2b^2)$ and $-1/(2b^2)$. This finite peak is not a positive probability mass at $\mu$. Indeedso $\mathbb{P}(X=\mu )=0$, just as for any absolutely continuous probability distribution.

The figure compares a Laplace distribution with $\mu =0$, $b=1$ against a normal distribution with mean zero and standard deviation $\sigma =\sqrt{2}$: both have variance 2. The right panel uses a logarithmic scale for the two-sided tail probability $\mathbb{P}(|X|>r)$. Its straight Laplace distribution curve comes from $e^{-r}$; the normal distribution tail falls faster. Thus values far from the center can be more plausible under a Laplace distribution even after matching the variance. These are theoretical curves, not measured frequencies. The tails still have exponential decay; this is not a heavy-tailed distribution in the convention requiring every positive exponential moment to diverge.

Integrating either half of the probability density function gives the cumulative distribution functionIn particular $F(\mu )=1/2$. The basic moments areThus the scale is the expected value of the absolute distance from the center, and is not the standard deviation. ^[ref] A short way to obtain the moments is to note that $R=|X-\mu |$ has an exponential distribution with rate $1/b$: $\mathbb{P}(R>r)=e^{-r/b}$ for $r\ge 0$. Integrating gives $\mathbb{E}R=b$ and $\mathbb{E}{R}^2=2b^2$; the symmetric distribution gives the centered first moment zero.

The normal distribution is a probability distribution on the real line with mean $\mu$ and positive standard deviation $\sigma$. Its probability density function isIn the notation $X\sim N(\mu ,{\sigma }^2)$, the second parameter is the variance ${\sigma }^2$, not the standard deviation $\sigma$. The probability density function is symmetric about $\mu$, highest there, and integrates to one. These formulas and the parameter convention are given in MIT 18.05, Reading 5c, §5: normal distributions.

The normal distribution is a location-scale family: if $Z$ is a standard normal random variable, then $X=\mu +\sigma Z$. Changing $\mu$ moves the center; changing $\sigma$ stretches the horizontal distances from it. The reciprocal factor $1/\sigma$ lowers the probability density function as the curve widens, preserving total area. This is why one standard normal distribution suffices to calculate probabilities for the whole family. ^[ref].

The height $f_X(x)$ is a probability density function value, not $\mathbb{P}(X=x)$. For $a\le b$, the probability $\mathbb{P}(a\le X\le b)={\int }_a^b{f}_X(x)dx$ is an area, and $\mathbb{P}(X=x)=0$ for every individual $x$. If $X$ is measured in some unit, $\mu$ and $\sigma$ use that unit, ${\sigma }^2$ uses its square, and $f_X$ uses its reciprocal; the integral is a dimensionless quantity. A change of measurement unit therefore changes the height without changing the probability. The area interpretation is developed in MIT 18.05, Reading 5b: continuous random variables.

The usefulness of the normal distribution goes beyond its simple shape. Under the central limit theorem, sums of independent and identically distributed random variables with finite positive variance, after centering and scaling, approach the standard normal distribution. Thus approximations by a normal distribution can arise even when the individual random variables have quite different probability distributions. The hypotheses matter: a model of measurement error still needs evidence that this approximation is appropriate. ^[ref]. Exact closure under a linear combination of independent normal random variables provides another reason the family is convenient.

Historically, the curve appeared in a different problem from fitting measurements. In the 1756 edition of The Doctrine of Chances, Abraham de Moivre reproduces work he dates to 12 November 1733 on approximating sums of binomial distribution probabilities for equally likely success and failure; page 245 already gives the one-sided central area as $0.341344$. ^[ref]. In his 1809 Theoria motus, Carl Friedrich Gauss derived a normal distribution for measurement error from a condition on the arithmetic mean, then connected maximizing its likelihood to minimizing the sum of squared errors under that model, the principle behind ordinary least squares. These are distinct contributions: approximating binomial distribution probabilities and justifying an estimation rule. ^[ref].

For a short normalization check, substitute $z=(x-\mu )/\sigma$ in the integral of $f_X$. The Gaussian integral gives ${\int }_{\mathbb{R}}{e}^{-z^2/2}dz=\sqrt{2\pi }$, so the total area is one. The standard normal density is an even function, giving $\mathbb{E}[Z]=0$; integration by parts gives $\mathbb{E}[Z^2]=1$. Then linearity of expectation and the variance scaling rule applied to $X=\mu +\sigma Z$ yield mean $\mu$ and variance ${\sigma }^2$.

A finite-dimensional exponential family has common dominating measure and densityHere $\eta$ is the natural parameter of an exponential family, and normalization determines the cumulant function of an exponential family $\kappa$. A full regular family lets $\eta$ vary over an open natural parameter domain. A curved exponential family restricts $\eta$ to a lower-dimensional submanifold, while a minimal representation removes redundant affine combinations of the statistic.

Differentiating the normalization under regularity givesConsequently $\kappa$ is convex and its Hessian is the Fisher information matrix in natural coordinates. For an independent size-$n$ sample, $S={\sum }_{i}T(Y_i)$ is a sufficient statistic: the joint density factors into ${\prod }_{i}h(y_i)$ times $\exp \{{\eta }^{\mathsf{T}}S-n\kappa (\eta )\}$, so the conditional law given $S$ loses its parameter dependence. An interior maximum-likelihood estimator solvesmatching empirical and theoretical sufficient-statistic means. Under minimality the Hessian is positive definite, so the log-likelihood is strictly concave; existence of an interior maximizer still needs the observed statistic to be in the appropriate interior convex support.

For a full open natural family, the canonical sufficient statistic is a complete statistic: if ${\mathbb{E}}_{\eta }g(S)=0$ throughout the open domain, exponential tilting about an interior point makes the Laplace transform of the integrable signed measure $g(s)P_{{\eta }_0}(ds)$ zero on a neighborhood. Transform uniqueness makes that measure zero, so $g(S)=0$ almost surely. This requires integrability under the family. It is not an automatic property of curved restrictions. The sufficient-statistic reduction, explicit information and cumulant identities make these families especially useful for exact and higher-order inference. Normal distributions, Poisson distributions and gamma distributions supply familiar examples.

A transformation model is built differently: a group acts on the sample and parameter spaces, and its probability laws satisfyIn a transitive model all laws are transforms of a fixed reference law. The location family is generated by translations, with density $f_{\mu }(y)=f_0(y-\mu )$. The positive-scale family has density $f_{\sigma }(y)={\sigma }^{-1}{f}_0(y/\sigma )$, and the location-scale family combines them asThe Jacobian factor is essential. For independent observations its power is ${\sigma }^{-n}$. The base density need not have moments, and a transformation model need not have fixed support.

An equivariant estimator respects the parameter action. With a well-defined equivariant fit $T(y)$ valued in the transformation group, the normalized sampleis a maximal invariant. Equivariance gives $a(gy)=a(y)$; if normalized samples agree, $y^{\prime }$ is obtained from $y$ by $T(y^{\prime })T(y{)}^{-1}$. In a transitive transformation model its law is parameter-free, so it is an ancillary statistic. For location and positive scale, centered standardized residuals provide this construction on nondegenerate samples. If the parameter action has a nontrivial stabilizer, normalization must retain its quotient action rather than pretending that a unique group element is determined.

This geometry motivates conditioning on an ancillary and using the P-star approximation for the fitted parameter. Ancillarity means parameter-free distribution; it does not by itself assert independence from the estimator. In a normal location model with known scale, the sample mean and centered residuals are independent by their joint Gaussian law and zero covariance. In an exponential scale model, the sample sum and its normalized proportions are independent, again an additional model property.

The two classes overlap but are distinct. A normal location-scale family is both an exponential family and a transformation model; an exponential mean family is both a one-dimensional natural exponential family and a scale model. A general Cauchy distribution location family is a transformation model without a fixed finite-dimensional regular exponential-family sufficient-statistic reduction. Exponential-family structure emphasizes sufficient statistics and natural-parameter convexity; transformation structure emphasizes equivariance, group orbits and ancillary reduction.

A transformation model consists of a group $G$ acting on the sample space and on the set of statistical parameters, with the compatibility propertyIn words, transforming a sample from $P_{\theta }$ produces a sample from $P_{g\theta }$. A transitive transformation model is generated from one reference distribution by the group action. For densities, the compatibility property includes the Jacobian determinant of the sample transformation; it is not merely equality of density values at transformed points.

A maximal invariant $A$ is constant on orbits of a group action and separates them: $A(gx)=A(x)$, and $A(x)=A(y)$ implies $y=gx$ for some $g$. An equivariant estimator $T$ of the parameter satisfies $T(gx)=gT(x)$. Equivariance means the estimate follows the transformed parameter, whereas invariance means the reported value stays unchanged.

First suppose the set of statistical parameters can be identified with the group, as happens for a simply transitive action. An equivariant estimator then provides a group-valued fit $\hat{g}(x)$ with $\hat{g}(gx)=g\hat{g}(x)$. Normalize the sample by this fit:Indeed $A(gx)=\hat{g}(x{)}^{-1}{g}^{-1}gx=A(x)$. Conversely, if $A(x)=A(y)$, then $y=\hat{g}(y)\hat{g}(x{)}^{-1}x$, placing $x,y$ in the same orbit of a group action. This establishes maximality, rather than only invariance.

For a transitive action with a nontrivial stabilizer $H$ of a reference parameter ${\theta }_0$, a parameter-valued equivariant estimator does not specify a unique normalizing transformation. Choose a section $h_{\theta }$ with $h_{\theta }{\theta }_0=\theta$ and form $b(x)=h_{T(x)}^{-1}x$. Under $g$,Thus the $H$-orbit of $b(x)$ is invariant. It is maximal: if $b(y)=hb(x)$ with $h\in H$, then $y=h_{T(y)}h{h}_{T(x)}^{-1}x$. Appropriate measurable sections and orbit of a group action coordinates are assumed when treating this as a statistic. This qualification explains why an arbitrary equivariant statistic alone is not automatically a complete set of invariant coordinates.

For the location-scale family, write $X_i=\mu +\sigma Z_i$ with $\sigma >0$ and a fixed joint distribution of $Z$. The group consists of maps $x_i\mapsto a+b{x}_i$ with $b>0$, acting on parameters by $(\mu ,\sigma )\mapsto (a+b\mu ,b\sigma )$. For a nonconstant sample of size $n\ge 2$, take the equivariant estimatorIt gives the maximal invariantIts coordinates satisfy ${\sum }_i{A}_i=0$ and ${\sum }_i{A}_i^2=n$. They stay unchanged under location and positive-scale transformations. If two samples have the same standardized vector, thenso a single group transformation maps the first sample onto the second. The vector retains the observation labels: sorting it would discard information not removed by this location-scale group. The construction is algebraic and does not require the underlying distribution to have finite moments. Constant samples can be assigned a separate invariant value; they form a single exceptional orbit of a group action and have probability zero under an independent continuous base distribution.

Finally, in a transitive transformation model every invariant statistic is ancillary. Choose $g$ taking ${\theta }_0$ to $\theta$ and use $X_{\theta }\stackrel{d}{=}g{X}_{{\theta }_0}$; invariance gives $A(X_{\theta })\stackrel{d}{=}A(X_{{\theta }_0})$. Thus the standardized residual configuration provides a natural conditioning variable, while the equivariant fit contains the location and scale information.

An ancillary statistic has a sampling distribution independent of the unknown parameter. It describes a realized aspect of the sample configuration without supplying a marginal likelihood for the parameter. Its observed value can nevertheless be crucial to the precision of inference from the rest of the data. In a transitive transformation model, invariance gives a common distribution under every parameter value, so maximal invariants provide natural ancillaries, such as the standardized configuration in a location-scale family.

Conditional inference uses the sampling distribution of an estimator or sufficient statistic conditional on the observed ancillary. With joint coordinates $(T,A)$, the factorizationshows that the conditional likelihood and the joint likelihood differ only by a parameter-free data factor. Conditional reference distributions, confidence intervals and p-values can still differ markedly from unconditional ones, because they adapt to the realized configuration. For a continuous ancillary one uses a regular conditional distribution or conditional density; dividing by the zero probability of a single ancillary value is not a valid definition.

An exact example makes this adaptation visible. Let independent observations be uniform on $(\theta -1/2,\theta +1/2)$, with $n\ge 2$. Put $U=X_{(1)}$, $V=X_{(n)}$, $R=V-U$ and $C=(U+V)/2$. Specifying the minimum and maximum contributes $n(n-1)$ choices of observations, while each remaining observation must lie between them. HenceThe transformation $u=c-r/2$, $v=c+r/2$ has unit absolute Jacobian determinant. Its support is $0<r<1$, $|c-\theta |<(1-r)/2$. Integrating over $c$ giveswhich is independent of $\theta$, proving that the range is ancillary. Dividing the joint density by this marginal density givesThe centered conditional pivot is uniform, sohas conditional coverage $1-\alpha$ at every $0<r<1$, and hence unconditional coverage $1-\alpha$ as well. A large realized range leaves little room for translating the interval and gives more precise inference. This is range-conditioned uniform location inference. The range and center are not independent: the conditional spread depends on the range, even though the range is ancillary.

Completeness can give independence in other models. Suppose $S$ is a complete sufficient statistic and $A$ is ancillary. For any bounded measurable $h$, sufficiency makes $g(S)=E_{\theta }(h(A)\mid S)$ a function independent of $\theta$, while ancillarity makes $c=E_{\theta }h(A)$ independent of $\theta$. Since $E_{\theta }(g(S)-c)=0$ for every $\theta$, completeness implies $g(S)=c$ almost surely. Taking all bounded indicator functions $h$ proves independence of $A$ and $S$. This is the content of Basu's theorem, with the proof displaying precisely where sufficiency and completeness are used. Without completeness the conclusion fails: in the uniform location example the minimum and maximum are sufficient, and contain the nondegenerate ancillary range.

There are practical qualifications. Ancillaries need not be unique, and some models supply only approximate ancillary coordinates; one must specify the conditioning surface and the accuracy intended. Conditioning on a statistic sufficient for a nuisance parameter is a different device: it may remove that nuisance even though the statistic is not ancillary for the full parameter. Conditioning indiscriminately can discard useful information. In higher-order likelihood inference, the ancillary coordinates specify how the fitted data are varied, entering both the p* density and the sample derivatives in the modified profile likelihood. Their role is therefore substantive, rather than merely a label for an extra statistic.

In a parametric statistical model $\{P_{\theta }:\theta \in \Theta \}$, the likelihood function $L(\theta ;y)$ measures the relative support given by the observed sample for different parameter values. Maximum likelihood estimation supplies a point estimate; the sampling distributions of suitable statistics calibrate significance tests and confidence intervals. A sufficient statistic retains all parameter-dependent information in the likelihood function, so one first seeks a reduction to such a statistic rather than discarding information arbitrarily.

The characteristic conditioning principle in Fisherian conditional inference is to calibrate inference conditional on the observed value of an ancillary statistic. With no nuisance parameter, a statistic $A$ is ancillary when its law is the same for every $\theta$. Its observed value can nevertheless describe the experimental configuration or precision. Conditional inference uses ${\mathcal{L}}_{\theta }(T\mid A=a)$ for an informative statistic $T$, with $a$ the observed ancillary value. Since the marginal law of $A$ does not depend on $\theta$, conditioning leaves the parameter-dependent likelihood factor unchanged, while potentially changing the appropriate repeated-sampling reference distribution. Conditioning on a continuous $A$ means using a regular conditional distribution, rather than dividing by $\mathbb{P}(A=a)$.

With parameters $(\psi ,\lambda )$, where $\psi$ is the target and $\lambda$ a nuisance parameter, a fully ancillary statistic has a law independent of both. A partial ancillary statistic for $\psi$ may have a law depending on $\lambda$ but not on $\psi$. To eliminate $\lambda$ by conditioning, one needs the actual conditional law of the informative statistic to be free of $\lambda$; partial ancillarity by itself does not guarantee this. For example, if independent counts have Poisson distributions with means $\lambda \psi$ and $\lambda (1-\psi )$, their total $N$ has Poisson distribution with mean $\lambda$, independent of $\psi$. The first count conditional on $N=m$ has binomial distribution with parameters $(m,\psi )$. Conditioning removes the nuisance intensity $\lambda$ exactly. More generally, nuisance-free pivotal quantities or conditional likelihoods can be used where available; there is no universal conditioning device that removes every nuisance parameter.

For a concrete example of non-uniqueness of maximal ancillary statistics, consider one observed pair $(U,V)\in \{0,1{\}}^2$, withBoth $U$ and $V$ have Bernoulli distribution with success probability $1/2$, independent of $\theta$, so each is an ancillary statistic. Their generated partitions are different and incomparable. Moreover, each is maximal: refining either two-point cell singles out a point whose probability is $\theta /2$ or $(1-\theta )/2$, which is parameter-dependent. Their joint statistic is not ancillary, since $P_{\theta }(U=V)=\theta$. Thus there is no common finer ancillary partition that reconciles the two choices. The conditional laws differ as labelled experiments: $P_{\theta }(V=U\mid U)=\theta$ and $P_{\theta }(U=V\mid V)=\theta$, but they condition on different observed information. This illustrates why “condition on a maximal ancillary” need not specify a unique procedure.

For the location-scale family, write $Y_i=\mu +\sigma Z_i$, where the $Z_i$ are independent with the fixed density $f_0$. Let $s_Z^2=n^{-1}{\sum }_i(Z_i-\bar{Z}{)}^2$. The transformation givesand thereforeFor $n\ge 2$, $s_Z>0$ almost surely, because an independent sample from a density has probability zero of all observations coinciding. The right side depends only on a sample from the fixed base law, so the entire vector $A$ has a law independent of $(\mu ,\sigma )$: $\boxed{A\text{ is ancillary}.}$ No finite population mean or variance is needed for this algebraic argument. With one observation the displayed standardization is undefined, so at least two observations are implicitly required.

The pair $(\hat{\mu },\hat{\sigma })$ together with $A$ reconstructs the full sample through $Y_i=\hat{\mu }+\hat{\sigma }{A}_i$. Thus conditional inference in a location-scale family useswhere $a$ is the observed residual configuration. Equivalently one can use the parameter-free conditional law of $((\hat{\mu }-\mu )/\sigma ,\hat{\sigma }/\sigma )$ given $A=a$. To see this explicitly, the centered residual subspace has dimension $n-1$, so its radial coordinate $t=\hat{\sigma }$ contributes a Jacobian determinant factor proportional to $t^{n-2}$. The conditional density of $\hat{\mu }=m$, $\hat{\sigma }=t>0$ is therefore proportional toSetting $u=(m-\mu )/\sigma$ and $v=t/\sigma$ gives a conditional density proportional to $v^{n-2}{\prod }_i{f}_0(u+v{a}_i)$, with no unknown parameter. The normalizing factor can depend on $a$, but not on $\mu$ or $\sigma$. These statements concern almost every attainable ancillary value.

A Q-Q plot is a scatter plot pairing quantiles at the same probability levels. For a statistical sample of size $n$, sort the observations into order statistics $x_{(1)}\le \cdots \le x_{(n)}$. Choose plotting positions $p_i$ and a reference quantile function $Q_0$, then plotHere the horizontal coordinate is a reference value, and the vertical coordinate is an observed value; neither axis is the probability $p_i$. The displayed plotting positions are our chosen convention, placing every level strictly between zero and one. They are not a claim that the $i$th order statistic equals that population quantile. For two statistical samples, use sample quantiles at matching levels, with a stated interpolation convention when their sizes differ. ^[ref]

As a computed example, take $n=19$ and the standard normal quantiles $z_i={\Phi }^{-1}(p_i)$, where $\Phi$ is the standard normal cumulative distribution function. Construct the ordered values $a_i=10+2z_i$. At ranks $1,10,19$, the plotting positions are $1/38,1/2,37/38$, the reference quantiles are approximately $-1.937932,0,1.937932$, and the constructed values are approximately $6.124137,10,13.875863$. Every point lies on $a=10+2z$ by construction. These 19 values are deliberately chosen, not random draws or real measurements.

The usefulness of a Q-Q plot is that it turns a comparison of probability distributions into geometry: if a reference random variable $Z$ is transformed into $X=\mu +\sigma Z$ with $\sigma >0$, its quantile function satisfies $Q_X(p)=\mu +\sigma Q_Z(p)$. This follows because a positive scaling preserves ordering. Thus the location-scale family produces a straight population Q-Q plot, and an approximate sample line suggests a compatible shape over the observed range. Its intercept describes location and its slope describes scale. The identity line requires the same location, scale and units on both axes; even then its displayed angle is 45 degrees only with equal axis scales. Curvature is often more informative than a histogram because it shows which quantiles depart from the reference. ^[ref]