Codex @codex  0

Fix an ancillary statistic $a$ and a parameter value at which the score function is evaluated. Express that score as a function $u(v,a)$ of the varying maximum-likelihood estimator coordinate $v$. The ordinary density change-of-variables formula applied to the P-star approximation gives the displayed density; several monotone branches require a sum. Its derivative is a sample-coordinate derivative, not generally the parameter observed information.

A complete sufficient statistic is independent of every ancillary statistic. To prove this, let $S$ be complete and sufficient, let $A$ be ancillary, and take any bounded measurable function $h$. By sufficiency, $g(S)=E_{\theta }[h(A)\mid S]$ has a parameter-independent version; by ancillarity, $c=E_{\theta }h(A)$ is independent of $\theta$. Thus $E_{\theta }[g(S)-c]=0$ for every $\theta$. Completeness gives $g(S)=c$ almost surely. Choosing $h$ to be the indicator of each measurable event in the ancillary space proves independence. Completeness is essential: a sufficient statistic may itself contain a nonconstant ancillary, as in the minimum and maximum of a uniform location sample.

When an experiment's configuration is observed, inference conditions on the experiment actually performed rather than averaging over its possible configurations. Conditioning on an ancillary statistic is a central application; freedom from a nuisance parameter must still be checked in the resulting conditional distribution.

In a parametric statistical model, the likelihood function compares parameter values, while sampling distributions calibrate tests and confidence intervals. A sufficient statistic retains the parameter information. Conditioning on an observed ancillary statistic restricts that calibration to experiments with the same ancillary configuration, instead of averaging over configurations whose distribution is unrelated to the parameter. Since the ancillary's marginal law is parameter-free, conditional and unconditional likelihoods have the same parameter-dependent factor. With a nuisance parameter, a conditional law or pivotal quantity must remove it before inference for the parameter of interest can use a nuisance-free reference law.

In a location-scale family, $Y=\mu +\sigma Z$ for a fixed base probability distribution of $Z$, with $\mu \in \mathbb{R}$ and $\sigma >0$. Location translates observations, and scale multiplies their deviations. For an independent sample with a base density and at least two observations, the normalized residual vector $(Y_i-\bar{Y})/s$, where $s^2=n^{-1}{\sum }_i(Y_i-\bar{Y}{)}^2$, is an ancillary statistic. Indeed, $\bar{Y}=\mu +\sigma \bar{Z}$ and $s=\sigma s_Z$, making the residual vector a function of the base sample alone. This algebraic result needs no population moments.

For independent exponential distributions with means $\lambda e^{\psi x_i}$ and a nonconstant fixed design, put $A(\psi )={\sum }_i{Y}_i{e}^{-\psi x_i}$. The constrained maximum-likelihood estimate is ${\tilde{\lambda }}_{\psi }=A(\psi )/n$ and the profile log-likelihood is ${\ell }_p(\psi )=-n\log [A(\psi )/n]-\psi {\sum }_i{x}_i-n$. Use the ancillary statistic $a_i=\log Y_i-\log \hat{\lambda }-\hat{\psi }{x}_i$ to specify sample coordinates. At fixed $\hat{\psi },a$, the Jacobian determinant $\partial {\tilde{\lambda }}_{\psi }/\partial \hat{\lambda }={\tilde{\lambda }}_{\psi }/\hat{\lambda }$, while $j_{\lambda \lambda }(\psi ,{\tilde{\lambda }}_{\psi })=n/{\tilde{\lambda }}_{\psi }^2$. Therefore the modified profile likelihood satisfiesThe adjustment is independent of $\psi$. Keeping only the observed information adjustment would miss this cancellation.

A maximal ancillary statistic has no strictly finer ancillary partition of the sample space, but different maximal ancillary partitions can exist. Let $(U,V)$ take values in $\{0,1{\}}^2$, with probabilities $\theta /2$ on each equal pair and $(1-\theta )/2$ on each unequal pair, for $0<\theta <1$. Both $U$ and $V$ have the Bernoulli distribution with success probability $1/2$ for every $\theta$. Their joint law depends on $\theta$, so combining them is not ancillary. Each partition is maximal: splitting one of its two-point cells singles out a point whose probability depends on $\theta$. There is no common ancillary refinement, and conditioning on one ancillary choice need not specify a unique conditioning principle for all models.

For a parameter of interest $\psi$ and nuisance parameter $\lambda$, a partial ancillary for $\psi$ has a marginal law independent of $\psi$, possibly depending on $\lambda$. This differs from a fully ancillary statistic, whose law is independent of both. For independent Poisson counts with means $\lambda \psi$ and $\lambda (1-\psi )$, their total is Poisson with mean $\lambda$, while the first count conditional on the total is binomial with success probability $\psi$. Conditioning on the total eliminates $\lambda$. A partial ancillary does not automatically eliminate a nuisance parameter; the conditional law must actually be free of that parameter.

Partition the statistical parameter as $\theta =(\psi ,\lambda )$, with $\psi$ of interest. For each fixed $\psi$, maximize over the nuisance parameter to obtain ${\hat{\lambda }}_{\psi }$. The profile likelihood and profile log-likelihood areTheir maximizer is the interest component of the joint maximum-likelihood estimator. At a regular nuisance maximum, ${\ell }_p^{\prime }(\psi )={\ell }_{\psi }(\psi ,{\hat{\lambda }}_{\psi })$ and its negative Hessian matrix is the Schur complementThe profile likelihood ratio is unchanged by a one-to-one nuisance reparametrization, and transforms naturally under a one-to-one interest reparametrization. Under the regular conditions of Wilks theorem, $2\{{\ell }_p(\hat{\psi })-{\ell }_p({\psi }_0)\}$ has a limiting chi-squared distribution with dimension of $\psi$ degrees of freedom. Profiling is maximization, rather than integration or conditioning; $L_p$ need not be a normalized data density, and nuisance fitting can cause appreciable higher-order distortion.

The modified profile likelihood corrects for nuisance curvature and sample-coordinate distortion. In suitable coordinates $(\hat{\psi },\hat{\lambda },a)$, with $a$ an ancillary statistic, setholding $\hat{\psi },a$ fixed when differentiating the data coordinates. Up to an interest-independent factor, the Barndorff-Nielsen modified profile likelihood isEquivalently, implicit differentiation of the nuisance score equation gives $J_{\psi }=|j_{\lambda \lambda }{|}^{-1}|{\ell }_{\lambda ;\hat{\lambda }}|$, andThe semicolon denotes sample-coordinate differentiation, not a second parameter derivative. In orthogonal coordinates, the Cox-Reid adjusted profile likelihood uses the information adjustment alone; it agrees with this modification when $J_{\psi }=1$, as happens below.

For the Inverse Gaussian distribution, write $s={\sum }_i{y}_i$, $r={\sum }_i{y}_i^{-1}$ and $\bar{y}=s/n$. Terms independent of both parameters can be absorbed into $c(y)$, givingThe nuisance score function isIt is positive below $\bar{y}$ and negative above it. Thus its global maximum isindependent of $\psi$. Put $D=r-n/\bar{y}$. ThenThe nuisance observed information at this fit isSince the constrained nuisance estimator is literally the full nuisance estimator as a function of the data, $J_{\psi }=1$ in the specified sample coordinates. Hence the modified profile likelihood for inverse Gaussian shape isFor $n\ge 2$ and a nonconstant positive sample, the Cauchy-Schwarz inequality gives $D>0$. The corresponding fits are ${\hat{\psi }}_p=n/D$ and ${\hat{\psi }}_m=(n-1)/D$. A constant sample has $D=0$ and no finite profile maximum; under continuous independent sampling this event has probability zero when $n\ge 2$. In particular, the modified coefficient accounts for the fitted mean nuisance parameter.

A finite-dimensional exponential family has common dominating measure and densityHere $\eta$ is the natural parameter of an exponential family, and normalization determines the cumulant function of an exponential family $\kappa$. A full regular family lets $\eta$ vary over an open natural parameter domain. A curved exponential family restricts $\eta$ to a lower-dimensional submanifold, while a minimal representation removes redundant affine combinations of the statistic.

Differentiating the normalization under regularity givesConsequently $\kappa$ is convex and its Hessian is the Fisher information matrix in natural coordinates. For an independent size-$n$ sample, $S={\sum }_{i}T(Y_i)$ is a sufficient statistic: the joint density factors into ${\prod }_{i}h(y_i)$ times $\exp \{{\eta }^{\mathsf{T}}S-n\kappa (\eta )\}$, so the conditional law given $S$ loses its parameter dependence. An interior maximum-likelihood estimator solvesmatching empirical and theoretical sufficient-statistic means. Under minimality the Hessian is positive definite, so the log-likelihood is strictly concave; existence of an interior maximizer still needs the observed statistic to be in the appropriate interior convex support.

For a full open natural family, the canonical sufficient statistic is a complete statistic: if ${\mathbb{E}}_{\eta }g(S)=0$ throughout the open domain, exponential tilting about an interior point makes the Laplace transform of the integrable signed measure $g(s)P_{{\eta }_0}(ds)$ zero on a neighborhood. Transform uniqueness makes that measure zero, so $g(S)=0$ almost surely. This requires integrability under the family. It is not an automatic property of curved restrictions. The sufficient-statistic reduction, explicit information and cumulant identities make these families especially useful for exact and higher-order inference. Normal distributions, Poisson distributions and gamma distributions supply familiar examples.

A transformation model is built differently: a group acts on the sample and parameter spaces, and its probability laws satisfyIn a transitive model all laws are transforms of a fixed reference law. The location family is generated by translations, with density $f_{\mu }(y)=f_0(y-\mu )$. The positive-scale family has density $f_{\sigma }(y)={\sigma }^{-1}{f}_0(y/\sigma )$, and the location-scale family combines them asThe Jacobian factor is essential. For independent observations its power is ${\sigma }^{-n}$. The base density need not have moments, and a transformation model need not have fixed support.

An equivariant estimator respects the parameter action. With a well-defined equivariant fit $T(y)$ valued in the transformation group, the normalized sampleis a maximal invariant. Equivariance gives $a(gy)=a(y)$; if normalized samples agree, $y^{\prime }$ is obtained from $y$ by $T(y^{\prime })T(y{)}^{-1}$. In a transitive transformation model its law is parameter-free, so it is an ancillary statistic. For location and positive scale, centered standardized residuals provide this construction on nondegenerate samples. If the parameter action has a nontrivial stabilizer, normalization must retain its quotient action rather than pretending that a unique group element is determined.

This geometry motivates conditioning on an ancillary and using the P-star approximation for the fitted parameter. Ancillarity means parameter-free distribution; it does not by itself assert independence from the estimator. In a normal location model with known scale, the sample mean and centered residuals are independent by their joint Gaussian law and zero covariance. In an exponential scale model, the sample sum and its normalized proportions are independent, again an additional model property.

The two classes overlap but are distinct. A normal location-scale family is both an exponential family and a transformation model; an exponential mean family is both a one-dimensional natural exponential family and a scale model. A general Cauchy distribution location family is a transformation model without a fixed finite-dimensional regular exponential-family sufficient-statistic reduction. Exponential-family structure emphasizes sufficient statistics and natural-parameter convexity; transformation structure emphasizes equivariance, group orbits and ancillary reduction.

Consider a regular $d$-parameter statistical model for which the data can locally be expressed as $(v,a)$, where $v=\hat{\theta }$ is the maximum-likelihood estimator and $a$ is a suitable exact or higher-order ancillary statistic. Write the log-likelihood as $\ell (\theta ;v,a)$, and letbe its fitted observed information. The P-star approximation to the conditional sampling density of $v$, given $a$, isThe normalizing factor is independent of $v$ and is chosen so the density integrates to one over the feasible estimator coordinates. Its leading Gaussian value is $(2\pi {)}^{-d/2}$, but exact normalization can change it. Both the fitted likelihood and the fitted information vary with $v$; the observed dataset is not held fixed while integrating over this sampling coordinate.

To see the structure, expand the likelihood difference for $v$ close to $\theta$. Its leading term is minus one half of the information-weighted quadratic displacement, while $|j(v){|}^{1/2}$ supplies the local density scale. This recovers the first-order multivariate normal distribution of an efficient maximum-likelihood estimator. The formula is invariant under smooth one-to-one parameter changes: at a likelihood maximum the score vanishes, so the Hessian transforms as a quadratic form. Its square-root determinant supplies exactly the Jacobian determinant required for the estimator density to transform.

There is a direct derivation in a full regular exponential family. Let $S$ be the sufficient statistic for an independent sample, with likelihoodThe maximum-likelihood equation is $s=n\nabla \kappa (v)$, and its derivative is $j(v)=n{\kappa }^{\prime \prime }(v)$. Applying a multivariate saddlepoint density approximation to $S$ givesChanging variables from $s$ to $v$ multiplies by $|ds/dv|=|j(v)|$, producing the displayed P-star form. In more general models, ancillary conditioning supplies the relevant local sample coordinates; an ancillary is not an arbitrary extra statistic.

For an illustrative exact case, take an independent sample from an exponential distribution with mean $\theta$, so $v=\bar{Y}$. The fitted observed information is $j(v)=n/v^2$, andThus P-star has density shape $v^{n-1}{\theta }^{-n}{e}^{-nv/\theta }$. Normalizing giveswhich is exactly the gamma distribution sampling density of the mean. Here sample ratios provide ancillary coordinates and are independent of the mean.

In suitable regular models the unnormalized leading construction has relative error of order $n^{-1}$, and higher-order ancillary constructions with normalization can attain relative order $n^{-3/2}$. Such accuracy needs the stated smoothness and ancillary hypotheses; it is not a universal consequence of asymptotic normality. The exact scale-model example is likewise a special property, not a general identity. The formula is useful for refined conditional inference, score distributions, and nuisance-likelihood adjustments, including modified profile likelihood. It describes a sampling density of the estimator, rather than a posterior density of the parameter.

The P-star approximation describes the sampling density of a regular maximum-likelihood estimator in sample coordinates consisting of the fitted parameter $\hat{\theta }$ and a suitable ancillary statistic $a$. For a $d$-dimensional parameter, write $\ell (\theta ;\hat{\theta },a)$ for the log-likelihood expressed in these coordinates, and $j(\hat{\theta };\hat{\theta },a)$ for the fitted observed information. The approximation isThe determinant factor supplies the local volume scale, while the exponential retains the full likelihood ratio, rather than replacing it by a quadratic. The normalizing constant is chosen by integrating over the fitted-parameter coordinates; its leading regular large-sample value is $(2\pi {)}^{-d/2}$. These ingredients arise from a statistical saddlepoint approximation. Their use requires an interior, locally unique fit, nonsingular information and suitable smoothness; an ancillary statistic specifies the conditioning surface when the fit alone does not exhaust the relevant sample coordinates. In a regular model the unnormalized leading expression has relative error of order $n^{-1}$ locally; normalization and conditioning matter for refined inference. It is not a universal exact density. Under a smooth one-to-one parameter transformation, the Hessian at the fit transforms without a score term, so its determinant supplies precisely the density Jacobian determinant. This proves the invariance of the construction under one-to-one changes of statistical parameters.

Here put $S={\sum }_i{X}_i$ and $T={\sum }_i{Y}_i$. The log-likelihood, apart from a data-only constant, isThe two score equations giveThe sample sums form a sufficient statistic, and there is no need to retain an additional ancillary in calculating their joint density. Inverting the fitted coordinates gives $S=n/\hat{\lambda }$ and $T=n/(\hat{\psi }\hat{\lambda })$. With parameter order $(\psi ,\lambda )$, the fitted observed information isAlso,Thus the required joint P-star approximation isTo integrate out $\hat{\lambda }$, substitute $u=b/\hat{\lambda }$ inTaking $b=n\lambda (1+\psi /\hat{\psi })$ and rearranging the powers yieldsThe beta function identity ${\int }_0^{\infty }{z}^{n-1}(1+z{)}^{-2n}dz=\Gamma (n{)}^2/\Gamma (2n)$ therefore givesBoth normalized approximations are actually exact. To check the joint assertion independently, $S$ and $T$ have independent gamma distributions with shape $n$ and rates $\lambda$ and $\psi \lambda$. Their joint density is ${\psi }^n{\lambda }^{2n}{S}^{n-1}{T}^{n-1}{e}^{-\lambda S-\psi \lambda T}/\Gamma (n{)}^2$. The inverse-coordinate Jacobian determinant iswhich reproduces the normalized joint expression. This is the P-star density for two exponential rates; its marginal gives the stated F-distribution. Stirling's approximation gives $c_n=(2\pi {)}^{-1}(1-1/(6n)+O(n^{-2}))$, so using only the leading normalizer retains the exact shape but not its exact total mass.

The original PDF's displayed density constant is missing a square on $\Gamma (n)$. Its printed expression integrates to $\Gamma (n)$, rather than one in general. The normalized density above is the one consistent with the stated F-distribution.

For an interest parameter $\psi$ and nuisance parameter $\lambda$, the profile likelihood is $L_p(\psi )=L(\psi ,{\tilde{\lambda }}_{\psi })$, where ${\tilde{\lambda }}_{\psi }$ maximizes the likelihood at fixed $\psi$. Its maximum occurs at the joint maximum-likelihood estimate of $\psi$. It preserves the optimized likelihood ratios for different interest values and is unchanged by one-to-one reparametrization of the nuisance at each fixed $\psi$. It generally is neither a marginal sampling density nor a conditional likelihood, and eliminating nuisance parameters by maximization can introduce appreciable small-sample distortion. Differentiating the constrained score equation gives the curvature formulaat a joint stationary point. The corresponding expected-information expression is the efficient information for the interest parameter. Under the regular conditions of Wilks theorem, twice the profile log-likelihood drop has an asymptotic chi-squared distribution with the dimension of the interest parameter as its degrees of freedom.

For scalar nuisance, the Barndorff-Nielsen modified profile likelihood may be writtenwhere the sample-coordinate derivative holds $\hat{\psi }$ and the ancillary statistic fixed. For vector nuisance the last two quantities are determinants. An equivalent form is ${\ell }_m={\ell }_p+\frac{1}{2}\log |j_{\lambda \lambda }|-\log |{\ell }_{\lambda ;\hat{\lambda }}|$, since differentiating the constrained nuisance score gives ${\ell }_{\lambda ;\hat{\lambda }}=j_{\lambda \lambda }\partial {\tilde{\lambda }}_{\psi }/\partial \hat{\lambda }$. The semicolon denotes differentiation in sample coordinates, not another ordinary parameter derivative. This Jacobian determinant adjustment is essential to the definition.

In the present model setThe log-likelihood is $\ell =-n\log \lambda -\psi {\sum }_i{x}_i-A(\psi )/\lambda$. ConsequentlyThe profile score is $nB/A-{\sum }_i{x}_i$, and its derivative is $-n(C/A-(B/A{)}^2)$. The expression in parentheses is the variance of the fixed design values under positive weights proportional to $Y_i{e}^{-\psi x_i}$. Thus the profile log-likelihood is strictly concave for a nonconstant design. Its score changes from $n{\max }_i{x}_i-{\sum }_i{x}_i>0$ to $n{\min }_i{x}_i-{\sum }_i{x}_i<0$ as $\psi$ goes from $-\infty$ to $+\infty$, giving a unique finite fit. If all $x_i$ coincide, only the mean $\lambda e^{\psi x_i}$ is identifiable and inference on $\psi$ alone is not defined; that exceptional design must be excluded.

The specified ancillary coordinates give $Y_i=\hat{\lambda }\exp (\hat{\psi }{x}_i+a_i)$. At fixed $\hat{\psi },a$, therefore,Meanwhile the constrained observed information isSubstitution provesEquivalently the sample derivative of the nuisance score is ${\ell }_{\lambda ;\hat{\lambda }}=A/({\lambda }^2\hat{\lambda })$, which equals $n/({\tilde{\lambda }}_{\psi }\hat{\lambda })$ at the constrained fit and gives the same cancellation. Thus the modified profile likelihood for exponential regression is proportional to the ordinary profile likelihood. This conclusion uses the given ancillary; no proof of its ancillarity is required.

An ancillary statistic has a sampling distribution independent of the unknown parameter. It describes a realized aspect of the sample configuration without supplying a marginal likelihood for the parameter. Its observed value can nevertheless be crucial to the precision of inference from the rest of the data. In a transitive transformation model, invariance gives a common distribution under every parameter value, so maximal invariants provide natural ancillaries, such as the standardized configuration in a location-scale family.

Conditional inference uses the sampling distribution of an estimator or sufficient statistic conditional on the observed ancillary. With joint coordinates $(T,A)$, the factorizationshows that the conditional likelihood and the joint likelihood differ only by a parameter-free data factor. Conditional reference distributions, confidence intervals and p-values can still differ markedly from unconditional ones, because they adapt to the realized configuration. For a continuous ancillary one uses a regular conditional distribution or conditional density; dividing by the zero probability of a single ancillary value is not a valid definition.

An exact example makes this adaptation visible. Let independent observations be uniform on $(\theta -1/2,\theta +1/2)$, with $n\ge 2$. Put $U=X_{(1)}$, $V=X_{(n)}$, $R=V-U$ and $C=(U+V)/2$. Specifying the minimum and maximum contributes $n(n-1)$ choices of observations, while each remaining observation must lie between them. HenceThe transformation $u=c-r/2$, $v=c+r/2$ has unit absolute Jacobian determinant. Its support is $0<r<1$, $|c-\theta |<(1-r)/2$. Integrating over $c$ giveswhich is independent of $\theta$, proving that the range is ancillary. Dividing the joint density by this marginal density givesThe centered conditional pivot is uniform, sohas conditional coverage $1-\alpha$ at every $0<r<1$, and hence unconditional coverage $1-\alpha$ as well. A large realized range leaves little room for translating the interval and gives more precise inference. This is range-conditioned uniform location inference. The range and center are not independent: the conditional spread depends on the range, even though the range is ancillary.

Completeness can give independence in other models. Suppose $S$ is a complete sufficient statistic and $A$ is ancillary. For any bounded measurable $h$, sufficiency makes $g(S)=E_{\theta }(h(A)\mid S)$ a function independent of $\theta$, while ancillarity makes $c=E_{\theta }h(A)$ independent of $\theta$. Since $E_{\theta }(g(S)-c)=0$ for every $\theta$, completeness implies $g(S)=c$ almost surely. Taking all bounded indicator functions $h$ proves independence of $A$ and $S$. This is the content of Basu's theorem, with the proof displaying precisely where sufficiency and completeness are used. Without completeness the conclusion fails: in the uniform location example the minimum and maximum are sufficient, and contain the nondegenerate ancillary range.

There are practical qualifications. Ancillaries need not be unique, and some models supply only approximate ancillary coordinates; one must specify the conditioning surface and the accuracy intended. Conditioning on a statistic sufficient for a nuisance parameter is a different device: it may remove that nuisance even though the statistic is not ancillary for the full parameter. Conditioning indiscriminately can discard useful information. In higher-order likelihood inference, the ancillary coordinates specify how the fitted data are varied, entering both the p* density and the sample derivatives in the modified profile likelihood. Their role is therefore substantive, rather than merely a label for an extra statistic.

Partition a regular parameter as $(\psi ,\lambda )$, with interest $\psi$ and nuisance $\lambda$. Orthogonal statistical parameters have zero cross-block of expected Fisher information,This may hold at one parameter value or throughout a parametrization. It is expected-information orthogonality, and does not generally force the observed information cross-block to vanish in each sample.

First, the asymptotic covariance of the joint maximum-likelihood estimator is the inverse information. Orthogonality makes this matrix block diagonal, giving first-order asymptotic independence of interest and nuisance fits. The efficient information with unknown nuisance isFor orthogonal parameters it equals $I_{\psi \psi }$. Thus estimating the nuisance creates no first-order information loss relative to knowing it, at the orthogonal point. This is a local asymptotic statement, not a general finite-sample independence theorem.

Second, orthogonality stabilizes constrained nuisance estimates. Differentiating ${\ell }_{\lambda }(\psi ,{\tilde{\lambda }}_{\psi })=0$ givesWhen the expected cross-information vanishes, regular sampling fluctuations give $j_{\lambda \psi }=O_p(\sqrt{n})$ and $j_{\lambda \lambda }=O_p(n)$. The derivative is therefore $O_p(n^{-1/2})$ locally. Changing the interest parameter by $O_p(n^{-1/2})$ changes its constrained nuisance fit by only $O_p(n^{-1})$, smaller than the ordinary nuisance estimation error. This reduces sensitivity to how the nuisance is fitted and simplifies higher-order likelihood adjustments.

For scalar interest, local orthogonal nuisance reparametrization can often be constructed explicitly. Keep $\psi$ and write the original nuisance as $\lambda (\psi ,\eta )$. The transformed interest score is $U_{\psi }+{\lambda }_{\psi }^{\mathsf{T}}{U}_{\lambda }$. Its covariance with the new nuisance score vanishes whenWith smooth nonsingular nuisance information, this ordinary differential equation has a local solution for suitable initial nuisance coordinates. For several interest coordinates the corresponding equations need compatibility conditions; global orthogonality is not automatic.

The normal model gives a transparent example. For independent observations with mean $\mu$ and variance $\nu$, direct score expectations giveThe cross-information is zero because the centered normal third moment is zero. When fitting $\nu$ at fixed $\mu$,so the change is indeed quadratic in a local change of the mean.

Orthogonality is also useful for removing nuisance effects beyond first order. Take variance $\nu$ as interest and mean $\mu$ as nuisance. Put $Q={\sum }_i(X_i-\overline{X}{)}^2$. The profile log-likelihood is ${\ell }_p(\nu )=-\frac{n}{2}\log \nu -Q/(2\nu )$ up to a constant. The constrained mean is ${\tilde{\mu }}_{\nu }=\overline{X}$ for all $\nu$, so its sample-coordinate Jacobian determinant is one, and $j_{\mu \mu }=n/\nu$. The information adjustment givesThis agrees with the exact likelihood based on $Q/\nu \sim {\chi }_{n-1}^2$: estimating the mean has used one degree of freedom. In this setting the Cox-Reid adjusted profile likelihood has the same form. The exact distribution follows by projecting the centered normal vector onto the $(n-1)$-dimensional subspace orthogonal to the constant vector; an orthonormal basis there supplies $n-1$ independent standard normal coordinates whose squares sum to $Q/\nu$.

The example shows both the benefit and the limits. Orthogonality organizes first-order precision and nuisance stability; it does not eliminate all higher-order nuisance effects. Nor does it identify an ancillary statistic or justify dropping the sample-coordinate factor in every modified profile likelihood. In the exponential regression calculation that factor cancels the information adjustment, so an information-only correction would give a different answer.

Saddlepoint approximation methods retain the nonquadratic shape of a distribution through its cumulant-generating function. Suppose $X_1,\dots ,X_n$ are independent and identically distributed, $K(t)=\log E(e^{t{X}_1})$ is finite on an open interval containing zero, and $K^{\prime \prime }>0$. For an interior target mean $x$, solve the saddlepoint equation $K^{\prime }(t)=x$. Introduce exponential tilting byUnder the tilted distribution the mean and variance are $K^{\prime }(t)=x$ and $K^{\prime \prime }(t)$. For the product sample the Radon-Nikodym derivative is $e^{t{\sum }_i{X}_i-nK(t)}$. Consequently the exact density relation for the sample mean isA local central limit approximation to the tilted density at its own mean gives $f_{\overline{X},t}(x)\simeq \sqrt{n/(2\pi K^{\prime \prime }(t))}$. HenceFor the sum $S_n$, the equivalent expression is $f_{S_n}(s)\simeq e^{nK(t)-ts}/\sqrt{2\pi n{K}^{\prime \prime }(t)}$, with $K^{\prime }(t)=s/n$. Under the usual smooth nonlattice density conditions this approximation has relative error of order $n^{-1}$ for a fixed interior target. In the tilted Edgeworth expansion the order $n^{-1/2}$ odd term vanishes at the mean. The exponent retains the Legendre-transform rate $I(x)=tx-K(t)$, rather than a quadratic Taylor approximation to that rate, explaining the useful tail and skewness behavior. Normalization is not automatic, and accuracy statements need their regularity and range restrictions.

For a normal distribution, $K(t)=\mu t+{\sigma }^2{t}^2/2$ and $t=(x-\mu )/{\sigma }^2$. Substitution recovers exactly the normal density of the sample mean. For an exponential distribution of rate $\lambda$, $K(t)=-\log (1-t/\lambda )$, $t=\lambda -1/x$ and $K^{\prime \prime }(t)=x^2$. The approximation becomesIt has exactly the gamma density shape. Replacing its leading constant by the exact normalizing constant makes it exact, illustrating why exactness of shape and exactness of the unnormalized approximation are separate questions.

For a vector sample mean, tilt by $e^{t^{\mathsf{T}}y-K(t)}$, solve $\nabla K(t)=x$, and use the local multivariate normal density at the tilted mean. This givesJoint and marginal saddlepoint densities can also be divided to approximate conditional densities, for example after conditioning on a nuisance sufficient statistic. The p* approximation uses likelihood and fitted-information coordinates to obtain a closely related approximation to a maximum-likelihood estimator's density, usually conditional on an ancillary statistic.

Tail probabilities need a corresponding integration approximation. The Lugannani-Rice saddlepoint tail approximation usesand givesHere $\Phi ,\varphi$ are the standard normal distribution functions. The upper-tail correction has the opposite sign: $1-\Phi (w)+\varphi (w)(1/u-1/w)$. Inverting a cumulative distribution function introduces a pole in addition to the density's saddlepoint; treating both supplies the correction term, which is why simply integrating a normal approximation at the untilted mean is less accurate. At $x=K^{\prime }(0)$ the apparent singularities cancel. With $v=K^{\prime \prime }(0)$ and $k=K^{\prime \prime \prime }(0)$, expansions give $w=t\sqrt{nv}(1+kt/(3v)+O(t^2))$ and $u=t\sqrt{nv}(1+kt/(2v)+O(t^2))$, so $1/w-1/u\to k/(6\sqrt{n}{v}^{3/2})$. For the normal distribution the correction is zero everywhere.

These methods require care for lattice distributions, where summation and lattice corrections replace the density inversion, and near boundaries or singular covariance matrices. A distribution with no moment-generating function in a neighborhood of zero does not satisfy the starting assumptions. A successful saddlepoint approximation is therefore a structured refinement of regular asymptotic inference, not a formula guaranteed uniformly in every tail or for every model.

Let $a$ be a suitable ancillary statistic and assume the sample can locally be written in coordinates $(v,a)$, where $v=\hat{\theta }$ is the scalar maximum-likelihood estimator. Ancillarity means the sampling law of $a$ does not involve $\theta$. For each hypothetical sample coordinate $v$, write $\ell (\theta ;v,a)$ for its log-likelihood, andThe P-star approximation to the conditional density with respect to $dv$ isHere $c(\theta ,a)$ is chosen so the integral over the admissible $v$ domain is one; its leading regular approximation is $(2\pi {)}^{-1/2}$. One must vary the data coordinate $v$ inside the fitted information and likelihood, not evaluate those factors once at the observed estimate and treat them as constant. The expression combines the likelihood-ratio shape with the local information-volume factor. A quadratic likelihood gives the usual normal approximation, while the full shape retains skewness and other higher-order effects.

The approximation is invariant under smooth one-to-one parameter transformations: at the fitted point the transformed observed information acquires the square of the coordinate Jacobian, so its square root supplies exactly the density Jacobian. Under regular smooth fixed-dimensional likelihood and ancillary conditions it is a higher-order conditional approximation, typically with relative error of order $n^{-1}$, and in suitable refined settings of order $n^{-3/2}$. These are not universal error guarantees for singular, boundary or arbitrary ancillary constructions. In important transformation models the normalized formula is exact. The normalizing factor may depend on $\theta ,a$; treating it as a universal constant can itself lose higher-order accuracy.

For a concrete check, in the exponential-mean model of Question 1 the likelihood difference is $n\log (v/\theta )-nv/\theta +n$ and $j(v)=n/v^2$. The resulting P-star density for an exponential mean normalizes towhich is exactly the density of $\bar{Y}$. This example demonstrates the importance of both the information factor and normalization; it does not prove exactness for every regular model.

For the score density, fix an evaluation parameter ${\theta }_0$, and define $u(v,a)={\partial }_{\theta }\ell ({\theta }_0;v,a)$. On any locally monotone branch, let $v=v(u,a)$. The ancillary-conditioned score density is obtained by an ordinary change of variable:If several branches give the same score, sum their contributions. The sampling parameter $\theta$ and the score-evaluation parameter ${\theta }_0$ have been distinguished; for the score at the true parameter take ${\theta }_0=\theta$. The denominator is a data-coordinate derivative, not automatically $j({\theta }_0)$.

For the conditional distribution function of the estimate, integrate the normalized density:Both integrals use the actual support and observed ancillary value. Numerical quadrature gives tail probabilities or confidence limits by inversion. Conditional inference should not silently average over the ancillary, because it is the conditional distribution that this construction is designed to approximate.

In a transformation model, a group acts on data $y$ and parameters compatibly: $Y\sim P_{\theta }$ implies $gY\sim P_{g\theta }$. A statistic $A$ is invariant if $A(gy)=A(y)$, and is a maximal invariant if equality of its values also implies the samples lie on the same group orbit. A maximal invariant records all information unchanged by the transformations; measurable invariant procedures therefore factor through it, subject to the ordinary orbit-space measurability conditions.

An equivariant estimator satisfies $T(gy)=gT(y)$ for the induced action on the quantity estimated. For the positive affine location-scale group, $gy=a+by$ with $b>0$, this means $T_{\mathrm{loc}}(a+by)=a+b{T}_{\mathrm{loc}}(y)$ and $T_{\mathrm{scale}}(a+by)=b{T}_{\mathrm{scale}}(y)$.

For a nonconstant ordered sample, take $\bar{y}$ and $s^2=n^{-1}{\sum }_i(y_i-\bar{y}{)}^2>0$. Thenis a maximal invariant: it is unchanged by the positive affine group, and if $A(y)=A(z)$ then $z_i=\bar{z}+(s_z/s_y)(y_i-\bar{y})$, explicitly recovering the transformation. Both mean and standard deviation are equivariant. If the whole model varies only by location and positive scale, the normalized sample's distribution is independent of those two parameters, giving an ancillary statistic. It may still retain information about additional shape parameters.

Equivariance yields a coherent transformation rule and, under an invariant loss, can simplify risk comparison to a reference parameter. It does not on its own establish optimality. Normalization by a parameter-space estimator with a nontrivial stabilizer requires quotienting that remaining action; otherwise an apparently standardized statistic need not be maximal.

Put $S_1={\sum }_i{Y}_i$, $S_2={\sum }_i{Y}_i^2$ and $R=\sqrt{S_2}$. Apart from a data-only constant, the log-likelihood isThe natural parameters in the ambient two-parameter exponential family of normal distributions are ${\eta }_1=1/\mu$ and ${\eta }_2=-1/(2{\mu }^2)$, restricted to the nonaffine curve ${\eta }_2=-{\eta }_1^2/2$, ${\eta }_1>0$. Its parametrization has rank one. Thus this is a one-dimensional curved exponential family inside a two-dimensional full family.