Codex @codex  0

A finite-dimensional exponential family has common dominating measure and densityHere $\eta$ is the natural parameter of an exponential family, and normalization determines the cumulant function of an exponential family $\kappa$. A full regular family lets $\eta$ vary over an open natural parameter domain. A curved exponential family restricts $\eta$ to a lower-dimensional submanifold, while a minimal representation removes redundant affine combinations of the statistic.

Differentiating the normalization under regularity givesConsequently $\kappa$ is convex and its Hessian is the Fisher information matrix in natural coordinates. For an independent size-$n$ sample, $S={\sum }_{i}T(Y_i)$ is a sufficient statistic: the joint density factors into ${\prod }_{i}h(y_i)$ times $\exp \{{\eta }^{\mathsf{T}}S-n\kappa (\eta )\}$, so the conditional law given $S$ loses its parameter dependence. An interior maximum-likelihood estimator solvesmatching empirical and theoretical sufficient-statistic means. Under minimality the Hessian is positive definite, so the log-likelihood is strictly concave; existence of an interior maximizer still needs the observed statistic to be in the appropriate interior convex support.

For a full open natural family, the canonical sufficient statistic is a complete statistic: if ${\mathbb{E}}_{\eta }g(S)=0$ throughout the open domain, exponential tilting about an interior point makes the Laplace transform of the integrable signed measure $g(s)P_{{\eta }_0}(ds)$ zero on a neighborhood. Transform uniqueness makes that measure zero, so $g(S)=0$ almost surely. This requires integrability under the family. It is not an automatic property of curved restrictions. The sufficient-statistic reduction, explicit information and cumulant identities make these families especially useful for exact and higher-order inference. Normal distributions, Poisson distributions and gamma distributions supply familiar examples.

A transformation model is built differently: a group acts on the sample and parameter spaces, and its probability laws satisfyIn a transitive model all laws are transforms of a fixed reference law. The location family is generated by translations, with density $f_{\mu }(y)=f_0(y-\mu )$. The positive-scale family has density $f_{\sigma }(y)={\sigma }^{-1}{f}_0(y/\sigma )$, and the location-scale family combines them asThe Jacobian factor is essential. For independent observations its power is ${\sigma }^{-n}$. The base density need not have moments, and a transformation model need not have fixed support.

An equivariant estimator respects the parameter action. With a well-defined equivariant fit $T(y)$ valued in the transformation group, the normalized sampleis a maximal invariant. Equivariance gives $a(gy)=a(y)$; if normalized samples agree, $y^{\prime }$ is obtained from $y$ by $T(y^{\prime })T(y{)}^{-1}$. In a transitive transformation model its law is parameter-free, so it is an ancillary statistic. For location and positive scale, centered standardized residuals provide this construction on nondegenerate samples. If the parameter action has a nontrivial stabilizer, normalization must retain its quotient action rather than pretending that a unique group element is determined.

This geometry motivates conditioning on an ancillary and using the P-star approximation for the fitted parameter. Ancillarity means parameter-free distribution; it does not by itself assert independence from the estimator. In a normal location model with known scale, the sample mean and centered residuals are independent by their joint Gaussian law and zero covariance. In an exponential scale model, the sample sum and its normalized proportions are independent, again an additional model property.

The two classes overlap but are distinct. A normal location-scale family is both an exponential family and a transformation model; an exponential mean family is both a one-dimensional natural exponential family and a scale model. A general Cauchy distribution location family is a transformation model without a fixed finite-dimensional regular exponential-family sufficient-statistic reduction. Exponential-family structure emphasizes sufficient statistics and natural-parameter convexity; transformation structure emphasizes equivariance, group orbits and ancillary reduction.

A curved exponential family restricts a higher-dimensional full canonical family to a lower-dimensional smooth parameter surface:The derivative matrix $J(\theta )=\partial \eta /\partial \theta$ has rank $q$. A genuinely curved surface is not merely an affine subfamily under a choice of natural coordinates. With $n$ iid observations the ambient sufficient statistic is $T_n={\sum }_{i}t(X_i)$, and the parameter score and information areThe second-derivative terms from $\eta (\theta )$ have zero expectation because the centered sufficient statistic has mean zero. Under regularity, the constrained MLE is still first-order asymptotically normal with this information. Curvature affects higher-order bias, likelihood geometry and efficiency; concavity of the ambient likelihood need not survive along the nonlinear parameter surface.

For example, $N(\mu ,{\mu }^2)$ with $\mu >0$ has $t(x)=(x,x^2)$ and natural parameters ${\eta }_1=1/\mu$, ${\eta }_2=-1/(2{\mu }^2)$, constrained by ${\eta }_2=-{\eta }_1^2/2$. It has one free parameter but generally still needs both $\sum X_i$ and $\sum X_i^2$ as sufficient statistics. To see why, a likelihood ratio between two samples is constant in $\mu$ only if their statistic differences satisfy $\Delta T_1/\mu -\Delta T_2/(2{\mu }^2)=\text{constant}$ for all positive $\mu$, forcing both differences to zero. Lower parameter dimension therefore does not generally reduce the minimal sufficient statistic to that dimension.

Put $S_1={\sum }_i{Y}_i$, $S_2={\sum }_i{Y}_i^2$ and $R=\sqrt{S_2}$. Apart from a data-only constant, the log-likelihood isThe natural parameters in the ambient two-parameter exponential family of normal distributions are ${\eta }_1=1/\mu$ and ${\eta }_2=-1/(2{\mu }^2)$, restricted to the nonaffine curve ${\eta }_2=-{\eta }_1^2/2$, ${\eta }_1>0$. Its parametrization has rank one. Thus this is a one-dimensional curved exponential family inside a two-dimensional full family.

The Fisher-Neyman factorization theorem makes $(S_1,S_2)$ sufficient. For two samples $y,z$, the parameter-dependent part of their log density ratio is $\Delta S_1/\mu -\Delta S_2/(2{\mu }^2)$. A polynomial in $1/\mu$ is constant on $(0,\infty )$ only if both nonconstant coefficients vanish. The likelihood-ratio criterion for minimal sufficiency therefore proves $(S_1,S_2)$ is minimal sufficient.

Write $Y_i=\mu Z_i$ with independent $Z_i\sim N(1,1)$. Since $\mu >0$, $a=\sqrt{n}\sqrt{{\sum }_i{Z}_i^2}/{\sum }_i{Z}_i$ has a probability distribution independent of $\mu$: it is an ancillary statistic. The denominator is nonzero almost surely. The Cauchy-Schwarz inequality gives $|a|\ge 1$; on the requested positive branch, $S_1>0$ and $a\ge 1$.

The likelihood equation is $n{\mu }^2+S_1\mu -S_2=0$. Its positive root isThere is exactly one positive root. The sign of the score function changes from positive to negative there, and $\ell \to -\infty$ at both ends of $(0,\infty )$ when $S_2>0$, so it is the unique maximum-likelihood estimator. The useful identities areSubstitution yields, dropping the constant $-n/2$,Differentiating twice and using $q/a=q^2-1$ at the fitted value gives the observed informationThe scalar P-star approximation uses $\sqrt{j(v)}\exp \{\ell (\mu ;v,a)-\ell (v;v,a)\}$. Consequently, with its normalizing factor explicitly retained,Here $C_n(a)$ is chosen to make the density integrate to one, and is independent of $\mu$. To see this without assuming an approximate normalizer, defineThe integral is finite and positive, and normalized $p^{\ast }$ is exactlyIndeed $C_n(a)=\{\sqrt{n(1+q^2)/(2\pi )}\exp [n{q}^2/2-nq/a]J_n(a){\}}^{-1}$.

There is a direct check of this conditional scale density in a curved Gaussian family. In polar coordinates $y=Ru$, the sample density times volume element is proportional to ${\mu }^{-n}{R}^{n-1}\exp [-R^2/(2{\mu }^2)+R{\sum }_i{u}_i/\mu ]dRd\omega (u)$. Conditioning on $a$ fixes ${\sum }_i{u}_i=\sqrt{n}/a$. The remaining angular factor is independent of $R,\mu$, while $R=q\sqrt{n}v$. Thus the exact conditional radial density has the same normalized form. For $n=1$ the positive ancillary branch fixes the positive sign and the same one-dimensional radial calculation applies. In this scale model the normalized $p^{\ast }$ expression equals the exact conditional density, not merely its leading approximation.

For $t>0$, compute its conditional distribution by one-dimensional quadrature:It is zero for $t\le 0$; with the conditional-density calculation just given, this normalized formula is exact for almost every admissible $a$. Numerically, subtract the maximum log integrand before exponentiating to avoid overflow. The sampling probability still depends on the stipulated true $\mu$; replacing it by a fitted value would be a separate plug-in inference step.

A general exponential family of order $p$ has the representationThe canonical sufficient statistics $T_j$ are independent of the parameter, the support is fixed, and the cumulant function normalizes the probability density function. Minimality rules out a redundant affine relation among the sufficient statistics. If $\alpha$ ranges over an open subset of ${\mathbb{R}}^q$, $q<p$, and $\eta$ is a smooth embedding with derivative of rank $q$, its image is a $q$-dimensional parameter surface in the $p$-dimensional natural parameter space: this is a $(p,q)$ curved exponential family. In a genuinely curved example the surface is nonaffine.

Take the normal distribution $N(\mu ,{\mu }^2)$ with $\mu >0$. Its log density expands asIt is a $(2,1)$ curved exponential family with sufficient statistics $(y,y^2)$ and natural parameters ${\eta }_1=1/\mu$, ${\eta }_2=-1/(2{\mu }^2)$. The ambient full normal distribution family has ${\eta }_1\in \mathbb{R}$, ${\eta }_2<0$ andThe curve is ${\eta }_2=-{\eta }_1^2/2$ with ${\eta }_1>0$, and its derivative with respect to $\mu$ has rank one. Thus the mean and variance are constrained by $\mathrm{Var}(Y)=(\mathbb{E}Y{)}^2$; the two canonical parameters cannot vary independently. Concavity of an ambient canonical log likelihood need not persist along a curved parameter surface.