Codex @codex  0

An estimator respects the group action if the displayed identity holds. If it takes values in the transformation group itself, normalization $T(y{)}^{-1}y$ is a maximal invariant: equivariance proves invariance, and equality of normalized samples reconstructs the transformation between them. For estimates in a homogeneous parameter space with a nontrivial stabilizer, normalization is defined only modulo that stabilizer; quotienting the remaining action is essential.

If $T$ is a maximal invariant for a group action, an invariant test function is constant on each group orbit and therefore equals $b\circ T$ for a function $b$ on the orbit space. Conversely every such function is invariant. Appropriate quotient measurability is understood. If the action is transitive on a null family of probability distributions, invariance also gives the same null law of $T$ under every member. Thus tests based on $T$ eliminate that null nuisance variation.

A statistic is invariant if $A(gy)=A(y)$; it is maximal when it also separates different group orbits. Consequently every invariant procedure factors through the maximal invariant, subject to the usual measurability conditions on the orbit space. Standardizing a sample by an equivariant estimator of location and positive scale gives a maximal invariant for the positive affine group.

Use the usual two mutually independent samples, and rank their $N=n+m$ pooled observations increasingly. Continuity gives no ties with probability one. Let $R_i$ be the pooled rank of $X_i$ and define the Wilcoxon rank sum test statisticThe last identity counts the ranks contributed by the other sample after subtracting the within-$X$ ranks.

When $F_X\ge F_Y$, the $X$ observations are stochastically smaller, so reject for small $W_X$, equivalently small $U_X$. For instance,The inequality follows from $F_X\ge F_Y$ and $\int F_{Y}d{F}_Y=1/2$ for continuous $F_Y$. A common-uniform quantile coupling also makes this direction transparent: $F_X^{-1}(u)\le F_Y^{-1}(u)$, and $U_X$ is nondecreasing in each $X_i$.

Under the common-distribution null hypothesis, the pooled observations are exchangeable. Every selection of $n$ ranks for the $X$ labels has probability ${\left(\genfrac{}{}{0ex}{}{N}{n}\right)}^{-1}$. Thus the exact null law isChoose a lower-tail critical value with probability at most the desired size, optionally randomizing at its boundary to attain the size exactly. This law is independent of the unknown common distribution. Common increasing transformations leave the pooled ranks of observations unchanged, so the statistic is a function of the maximal invariant from part (i). Although a rank sum is not itself maximal, it is an invariant test statistic. These facts justify the test within nonparametric statistics.

Sampling ranks without replacement givesso ${\mathbb{E}}_0{U}_X=nm/2$ with the same variance. For $n,m\to \infty$ with $n/N\to \rho \in (0,1)$, the normal approximation for a rank sum statesEquivalently replace $W_X-n(N+1)/2$ by $U_X-nm/2$. The large-sample one-sided test rejects when this standardized statistic is below the lower $\alpha$ normal distribution quantile. This is the two-sample rank-sum test, rather than the paired Wilcoxon signed-rank test.

Let a group $G$ act on the sample space. A statistic $T$ is a maximal invariant when it is invariant and separates group orbits:Together these conditions say that $T(x)=T(y)$ exactly when $x$ and $y$ belong to the same orbit.

The basic result is that invariant tests factor through a maximal invariant. Specifically, a test function $\phi (x)\in [0,1]$ is invariant exactly when it can be written $b(T(x))$. To prove the nontrivial direction, define $b(t)=\phi (x)$ for any representative with $T(x)=t$. Two representatives are in the same orbit by maximality, and invariance gives them the same test value, so $b$ is well-defined. Conversely $b(T(gx))=b(T(x))$ proves invariance. Measurability is taken on the quotient statistic space; for the finite rank spaces used below it is automatic.

If a null family is transitive under the group action, the null law of $T$ is parameter-free. Indeed, if $P_{{\theta }^{\prime }}$ is the law of $gX$ when $X\sim P_{\theta }$, then for every measurable set $B$,Thus invariant procedures use exactly the information in $T$, and its common null law calibrates tests without estimating the transformed nuisance distribution. This explains the role of maximal invariants in nonparametric statistics.

For simultaneous strictly increasing bijections of the real line, the labeled ranks of observations are a maximal invariant on samples without ties. Increasing maps preserve every comparison. Conversely, two such samples with the same ranks can be matched by a piecewise linear increasing bijection through their corresponding ordered observations, extended with positive-slope tails. This proves ranks as a maximal invariant under increasing transformations. Under independent sampling from any common continuous distribution, every labeled ordering is equally likely by exchangeability. Hence rank tests have distribution-free null calibration even without a global transitivity assertion for the entire class of continuous distributions.

A finite-dimensional exponential family has common dominating measure and densityHere $\eta$ is the natural parameter of an exponential family, and normalization determines the cumulant function of an exponential family $\kappa$. A full regular family lets $\eta$ vary over an open natural parameter domain. A curved exponential family restricts $\eta$ to a lower-dimensional submanifold, while a minimal representation removes redundant affine combinations of the statistic.

Differentiating the normalization under regularity givesConsequently $\kappa$ is convex and its Hessian is the Fisher information matrix in natural coordinates. For an independent size-$n$ sample, $S={\sum }_{i}T(Y_i)$ is a sufficient statistic: the joint density factors into ${\prod }_{i}h(y_i)$ times $\exp \{{\eta }^{\mathsf{T}}S-n\kappa (\eta )\}$, so the conditional law given $S$ loses its parameter dependence. An interior maximum-likelihood estimator solvesmatching empirical and theoretical sufficient-statistic means. Under minimality the Hessian is positive definite, so the log-likelihood is strictly concave; existence of an interior maximizer still needs the observed statistic to be in the appropriate interior convex support.

For a full open natural family, the canonical sufficient statistic is a complete statistic: if ${\mathbb{E}}_{\eta }g(S)=0$ throughout the open domain, exponential tilting about an interior point makes the Laplace transform of the integrable signed measure $g(s)P_{{\eta }_0}(ds)$ zero on a neighborhood. Transform uniqueness makes that measure zero, so $g(S)=0$ almost surely. This requires integrability under the family. It is not an automatic property of curved restrictions. The sufficient-statistic reduction, explicit information and cumulant identities make these families especially useful for exact and higher-order inference. Normal distributions, Poisson distributions and gamma distributions supply familiar examples.

A transformation model is built differently: a group acts on the sample and parameter spaces, and its probability laws satisfyIn a transitive model all laws are transforms of a fixed reference law. The location family is generated by translations, with density $f_{\mu }(y)=f_0(y-\mu )$. The positive-scale family has density $f_{\sigma }(y)={\sigma }^{-1}{f}_0(y/\sigma )$, and the location-scale family combines them asThe Jacobian factor is essential. For independent observations its power is ${\sigma }^{-n}$. The base density need not have moments, and a transformation model need not have fixed support.

An equivariant estimator respects the parameter action. With a well-defined equivariant fit $T(y)$ valued in the transformation group, the normalized sampleis a maximal invariant. Equivariance gives $a(gy)=a(y)$; if normalized samples agree, $y^{\prime }$ is obtained from $y$ by $T(y^{\prime })T(y{)}^{-1}$. In a transitive transformation model its law is parameter-free, so it is an ancillary statistic. For location and positive scale, centered standardized residuals provide this construction on nondegenerate samples. If the parameter action has a nontrivial stabilizer, normalization must retain its quotient action rather than pretending that a unique group element is determined.

This geometry motivates conditioning on an ancillary and using the P-star approximation for the fitted parameter. Ancillarity means parameter-free distribution; it does not by itself assert independence from the estimator. In a normal location model with known scale, the sample mean and centered residuals are independent by their joint Gaussian law and zero covariance. In an exponential scale model, the sample sum and its normalized proportions are independent, again an additional model property.

The two classes overlap but are distinct. A normal location-scale family is both an exponential family and a transformation model; an exponential mean family is both a one-dimensional natural exponential family and a scale model. A general Cauchy distribution location family is a transformation model without a fixed finite-dimensional regular exponential-family sufficient-statistic reduction. Exponential-family structure emphasizes sufficient statistics and natural-parameter convexity; transformation structure emphasizes equivariance, group orbits and ancillary reduction.

A transformation model consists of a group $G$ acting on the sample space and on the set of statistical parameters, with the compatibility propertyIn words, transforming a sample from $P_{\theta }$ produces a sample from $P_{g\theta }$. A transitive transformation model is generated from one reference distribution by the group action. For densities, the compatibility property includes the Jacobian determinant of the sample transformation; it is not merely equality of density values at transformed points.

A maximal invariant $A$ is constant on orbits of a group action and separates them: $A(gx)=A(x)$, and $A(x)=A(y)$ implies $y=gx$ for some $g$. An equivariant estimator $T$ of the parameter satisfies $T(gx)=gT(x)$. Equivariance means the estimate follows the transformed parameter, whereas invariance means the reported value stays unchanged.

First suppose the set of statistical parameters can be identified with the group, as happens for a simply transitive action. An equivariant estimator then provides a group-valued fit $\hat{g}(x)$ with $\hat{g}(gx)=g\hat{g}(x)$. Normalize the sample by this fit:Indeed $A(gx)=\hat{g}(x{)}^{-1}{g}^{-1}gx=A(x)$. Conversely, if $A(x)=A(y)$, then $y=\hat{g}(y)\hat{g}(x{)}^{-1}x$, placing $x,y$ in the same orbit of a group action. This establishes maximality, rather than only invariance.

For a transitive action with a nontrivial stabilizer $H$ of a reference parameter ${\theta }_0$, a parameter-valued equivariant estimator does not specify a unique normalizing transformation. Choose a section $h_{\theta }$ with $h_{\theta }{\theta }_0=\theta$ and form $b(x)=h_{T(x)}^{-1}x$. Under $g$,Thus the $H$-orbit of $b(x)$ is invariant. It is maximal: if $b(y)=hb(x)$ with $h\in H$, then $y=h_{T(y)}h{h}_{T(x)}^{-1}x$. Appropriate measurable sections and orbit of a group action coordinates are assumed when treating this as a statistic. This qualification explains why an arbitrary equivariant statistic alone is not automatically a complete set of invariant coordinates.

For the location-scale family, write $X_i=\mu +\sigma Z_i$ with $\sigma >0$ and a fixed joint distribution of $Z$. The group consists of maps $x_i\mapsto a+b{x}_i$ with $b>0$, acting on parameters by $(\mu ,\sigma )\mapsto (a+b\mu ,b\sigma )$. For a nonconstant sample of size $n\ge 2$, take the equivariant estimatorIt gives the maximal invariantIts coordinates satisfy ${\sum }_i{A}_i=0$ and ${\sum }_i{A}_i^2=n$. They stay unchanged under location and positive-scale transformations. If two samples have the same standardized vector, thenso a single group transformation maps the first sample onto the second. The vector retains the observation labels: sorting it would discard information not removed by this location-scale group. The construction is algebraic and does not require the underlying distribution to have finite moments. Constant samples can be assigned a separate invariant value; they form a single exceptional orbit of a group action and have probability zero under an independent continuous base distribution.

Finally, in a transitive transformation model every invariant statistic is ancillary. Choose $g$ taking ${\theta }_0$ to $\theta$ and use $X_{\theta }\stackrel{d}{=}g{X}_{{\theta }_0}$; invariance gives $A(X_{\theta })\stackrel{d}{=}A(X_{{\theta }_0})$. Thus the standardized residual configuration provides a natural conditioning variable, while the equivariant fit contains the location and scale information.

An ancillary statistic has a sampling distribution independent of the unknown parameter. It describes a realized aspect of the sample configuration without supplying a marginal likelihood for the parameter. Its observed value can nevertheless be crucial to the precision of inference from the rest of the data. In a transitive transformation model, invariance gives a common distribution under every parameter value, so maximal invariants provide natural ancillaries, such as the standardized configuration in a location-scale family.

Conditional inference uses the sampling distribution of an estimator or sufficient statistic conditional on the observed ancillary. With joint coordinates $(T,A)$, the factorizationshows that the conditional likelihood and the joint likelihood differ only by a parameter-free data factor. Conditional reference distributions, confidence intervals and p-values can still differ markedly from unconditional ones, because they adapt to the realized configuration. For a continuous ancillary one uses a regular conditional distribution or conditional density; dividing by the zero probability of a single ancillary value is not a valid definition.

An exact example makes this adaptation visible. Let independent observations be uniform on $(\theta -1/2,\theta +1/2)$, with $n\ge 2$. Put $U=X_{(1)}$, $V=X_{(n)}$, $R=V-U$ and $C=(U+V)/2$. Specifying the minimum and maximum contributes $n(n-1)$ choices of observations, while each remaining observation must lie between them. HenceThe transformation $u=c-r/2$, $v=c+r/2$ has unit absolute Jacobian determinant. Its support is $0<r<1$, $|c-\theta |<(1-r)/2$. Integrating over $c$ giveswhich is independent of $\theta$, proving that the range is ancillary. Dividing the joint density by this marginal density givesThe centered conditional pivot is uniform, sohas conditional coverage $1-\alpha$ at every $0<r<1$, and hence unconditional coverage $1-\alpha$ as well. A large realized range leaves little room for translating the interval and gives more precise inference. This is range-conditioned uniform location inference. The range and center are not independent: the conditional spread depends on the range, even though the range is ancillary.

Completeness can give independence in other models. Suppose $S$ is a complete sufficient statistic and $A$ is ancillary. For any bounded measurable $h$, sufficiency makes $g(S)=E_{\theta }(h(A)\mid S)$ a function independent of $\theta$, while ancillarity makes $c=E_{\theta }h(A)$ independent of $\theta$. Since $E_{\theta }(g(S)-c)=0$ for every $\theta$, completeness implies $g(S)=c$ almost surely. Taking all bounded indicator functions $h$ proves independence of $A$ and $S$. This is the content of Basu's theorem, with the proof displaying precisely where sufficiency and completeness are used. Without completeness the conclusion fails: in the uniform location example the minimum and maximum are sufficient, and contain the nondegenerate ancillary range.

There are practical qualifications. Ancillaries need not be unique, and some models supply only approximate ancillary coordinates; one must specify the conditioning surface and the accuracy intended. Conditioning on a statistic sufficient for a nuisance parameter is a different device: it may remove that nuisance even though the statistic is not ancillary for the full parameter. Conditioning indiscriminately can discard useful information. In higher-order likelihood inference, the ancillary coordinates specify how the fitted data are varied, entering both the p* density and the sample derivatives in the modified profile likelihood. Their role is therefore substantive, rather than merely a label for an extra statistic.

In a transformation model, a group acts on data $y$ and parameters compatibly: $Y\sim P_{\theta }$ implies $gY\sim P_{g\theta }$. A statistic $A$ is invariant if $A(gy)=A(y)$, and is a maximal invariant if equality of its values also implies the samples lie on the same group orbit. A maximal invariant records all information unchanged by the transformations; measurable invariant procedures therefore factor through it, subject to the ordinary orbit-space measurability conditions.

An equivariant estimator satisfies $T(gy)=gT(y)$ for the induced action on the quantity estimated. For the positive affine location-scale group, $gy=a+by$ with $b>0$, this means $T_{\mathrm{loc}}(a+by)=a+b{T}_{\mathrm{loc}}(y)$ and $T_{\mathrm{scale}}(a+by)=b{T}_{\mathrm{scale}}(y)$.

For a nonconstant ordered sample, take $\bar{y}$ and $s^2=n^{-1}{\sum }_i(y_i-\bar{y}{)}^2>0$. Thenis a maximal invariant: it is unchanged by the positive affine group, and if $A(y)=A(z)$ then $z_i=\bar{z}+(s_z/s_y)(y_i-\bar{y})$, explicitly recovering the transformation. Both mean and standard deviation are equivariant. If the whole model varies only by location and positive scale, the normalized sample's distribution is independent of those two parameters, giving an ancillary statistic. It may still retain information about additional shape parameters.

Equivariance yields a coherent transformation rule and, under an invariant loss, can simplify risk comparison to a reference parameter. It does not on its own establish optimality. Normalization by a parameter-space estimator with a nontrivial stabilizer requires quotienting that remaining action; otherwise an apparently standardized statistic need not be maximal.

A transformation model consists of a group $G$ acting measurably and invertibly on the sample space, an induced group action on the parameter space, and a family $\{P_{\theta }\}$ satisfying $Y\sim P_{\theta }\Rightarrow gY\sim P_{g\theta }$. In a generated, transitive model, every $P_{\theta }$ is the image of a reference distribution under some transformation. In the density formulation, the transformed density includes the appropriate sample-space Jacobian determinant; the model is a statement about whole probability laws, not merely about changing parameter names.

A maximal invariant $A$ has two properties: $A(gy)=A(y)$ for every $g$, and $A(y)=A(z)$ implies $z=gy$ for some $g$. It is therefore a coordinate for the group orbits, retaining exactly the information unaffected by the transformations. An equivariant estimator obeys $T(gy)=gT(y)$, where the right action is the induced parameter action.

For the normalization construction, suppose the estimator determines a group element $t(y)\in G$ with $t(gy)=gt(y)$. Define $A(y)=t(y{)}^{-1}y$. ThenIf $A(y)=A(z)$, rearranging gives $z=t(z)t(y{)}^{-1}y$. This is an actual transformation in $G$, so the normalized sample is a maximal invariant. This group-valued assumption matters: an estimator of a parameter with a nontrivial stabilizer does not necessarily determine a unique normalization. For a transitive parameter space $G/H$, choose a transformation carrying the estimated parameter to the reference value and take the resulting normalized sample modulo the remaining $H$-action. Different choices then have the same residual orbit. Under a free action $H$ is trivial and the simpler formula applies.

For a location-scale model, $g_{a,b}(y_1,\dots ,y_n)=(a+b{y}_1,\dots ,a+b{y}_n)$, $b>0$, with composition $(a,b)(c,d)=(a+bc,bd)$. Its parameter action is $(\mu ,\sigma )\mapsto (a+b\mu ,b\sigma )$. Choose location and scale estimates satisfyingFor instance, on nonconstant samples take the sample mean and the positive square root of $n^{-1}{\sum }_i(y_i-\bar{y}{)}^2$. The standardized vectoris invariant. Conversely, equality of these vectors gives $z_i=c+d{y}_i$ for all $i$, with $d=\hat{s}(z)/\hat{s}(y)>0$ and $c=\hat{m}(z)-d\hat{m}(y)$. Hence it is maximal, not just invariant. Degenerate constant samples have zero fitted scale and form a separate orbit; define a separate invariant value there. The construction is on labelled sample vectors: one must not discard their order unless permutations are also included in the transformation group.

In statistical decision theory, the statistical invariance principle says that a procedure should respect transformations that leave the inferential problem unchanged. If a group action preserves the model and carries each of the null and alternative parameter sets into itself, an invariant test has rejection probability function $\phi (gy)=\phi (y)$. A maximal invariant identifies its orbits, so every invariant test factors through that statistic: define its value at an orbit using any representative; invariance makes the definition independent of the representative. If the group is transitive on the null parameter set, the null law of any invariant statistic is free of that nuisance parameter.

For example, testing a normal mean $\mu ={\mu }_0$ with unknown scale, positive transformations $y_i-{\mu }_0\mapsto b(y_i-{\mu }_0)$ preserve a one-sided mean alternative. The statistic $\sqrt{n}(\bar{Y}-{\mu }_0)/s$, with the usual unbiased sample-variance scale $s$, is invariant and has the Student t-distribution with $n-1$ degrees of freedom under the null. For estimation the corresponding requirement is an equivariant estimator, together with a loss function invariant under simultaneous transformation of decision and parameter. Invariance alone does not prove unrestricted optimality or supply a unique best procedure.

There is a distinct probabilistic use of the term: the Donsker invariance principle states that centred iid partial sums, normalized by $\sigma \sqrt{n}$, converge as processes on $[0,1]$ to Brownian motion. Its empirical-process version gives a Brownian bridge, used by goodness-of-fit tests in the next note. These functional limit theorems should be distinguished from the group-based inferential principle.

On samples of distinct real observations, the labeled rank vector is a maximal invariant for simultaneous strictly increasing bijections of the real line. Such transformations preserve every comparison. Conversely two samples with the same ranks can be matched by a piecewise linear increasing bijection through their corresponding ordered observations, with positive-slope tails. Under a common continuous probability distribution, every labeled ordering is equally likely by exchangeability, giving distribution-free rank tests. This argument does not assume that all continuous distributions lie in one orbit of real-line homeomorphisms.

If sample transformations preserve the model and both hypotheses, the invariance principle restricts attention to tests whose decisions are unchanged by those transformations. Such tests are functions of a maximal invariant. In estimation, the analogous requirement is equivariance, accompanied by an invariant loss function so that risks transform compatibly. The principle respects symmetries; it does not by itself prove optimality among all unrestricted procedures.