For a sufficient statistic and parameter , a useful strong reduction requires the law of to be free of and the conditional distribution of given to be free of the nuisance parameter . In the discrete case it gives
where sums to one on each fiber, sums to one over , and sums to one over separately for every fixed . Ordinary ancillarity, or just the law of being free of , does not by itself guarantee that conditioning removes .
Bayesian network structure score Created 2026-10-05 Updated 2026-10-14
For observations and graph , Bayes' theorem gives . Here is the graph prior distribution and the local distribution statistical parameters. The integral is Bayesian model evidence; it averages over nuisance parameters with a proper statistical parameter prior distribution. For independent complete observations, the node-factorized likelihood function and independence of local statistical parameter prior distributions make the evidence factor over nodes; suitable conjugate priors can make the local integrals analytic. With missing node observations, integrating out unobserved values can couple the local parameters, so independence of local prior distributions alone does not guarantee this factorization. Proper priors and coherent hyperparameters matter for comparing different graphs.
Conditionality principle 2026-10-14
When an experiment's configuration is observed, inference conditions on the experiment actually performed rather than averaging over its possible configurations. Conditioning on an ancillary statistic is a central application; freedom from a nuisance parameter must still be checked in the resulting conditional distribution.
Conditional likelihood 2026-10-14
A conditional likelihood evaluates the conditional distribution of the observed data given a specified statistic. If that distribution does not involve a nuisance parameter, it permits inference about the remaining parameter without estimating the nuisance parameter. The conditioning statistic need not be ancillary; possible information loss must be assessed within the model. Conditional maximum likelihood maximizes this function.
Consider independent binomial counts under the common-probability null hypothesis, with fixed positive row sizes . Let , and assume . Conditional on , the probability mass function is the displayed multivariate hypergeometric distribution: in the joint likelihood, the factor is common to all admissible tables and cancels. The Vandermonde identity gives the denominator, so the conditional distribution is free of the unknown nuisance parameter .
For a table , its Pearson chi-squared statistic is
An exact conditional p-value sums the displayed probabilities over all tables with the same margins and . The ordering is by the Pearson chi-squared statistic, rather than by the probability of an individual table; different exact-test orderings can give different answers. Enumeration is practical when and the number of rows are small. Monte Carlo tests can approximate the same tail by sampling tables with fixed margins.[ref]
An exact arithmetic comparison is easy too. Let be the least common multiple of the row sizes. Compare the integer
since every table has the same positive denominator . This includes boundary ties without a floating-point tolerance. Conditional p-values are discrete and can be conservative; “exact” concerns their null calibration, not equality of rejection probability with every chosen significance level.
In a parametric statistical model, the likelihood function compares parameter values, while sampling distributions calibrate tests and confidence intervals. A sufficient statistic retains the parameter information. Conditioning on an observed ancillary statistic restricts that calibration to experiments with the same ancillary configuration, instead of averaging over configurations whose distribution is unrelated to the parameter. Since the ancillary's marginal law is parameter-free, conditional and unconditional likelihoods have the same parameter-dependent factor. With a nuisance parameter, a conditional law or pivotal quantity must remove it before inference for the parameter of interest can use a nuisance-free reference law.
The bridge from detector readout to a particle resonance is four-momentum conservation. In units , sum the measured final-state four-momenta:
For two photons separated by angle , gives the displayed diphoton formula. Two back-to-back photons of each have . A four-lepton candidate instead uses four four-momenta in the same sum. Collision debris outside the selected decay is not included. Since the partons carry unknown fractions of the incoming proton momenta, reconstructing the decay products is more informative than assigning the full beam energy to each candidate.
Figure 1.
Detector functions, reconstructed mass and a chosen resolution comparison
. Functional schematic followed by an original expectation model: sixty selected signal events at , a flat background of eight events per electronvolts, and chosen Gaussian mass widths and . No historical event counts or copied discovery histograms are shown. Details.
Suppose the intrinsic decay width is negligible compared with the stipulated detector spread. Let the reconstructed signal mass follow a normal distribution with mean and standard deviation . For a total expected selected signal and flat background density events per electronvolts, the expected count density is
This is a deliberately chosen response model, not a fit to either experiment. In the fixed window with , direct integration gives
The expected signal counts are for and for . Doubling the spread lowers the fraction in the same window without changing the total signal. Detector resolution can therefore change the visibility of a peak even with identical particle production.
As a separate counting example, stipulate an observed integer count and an exactly known background mean . The Poisson distribution gives a likelihood for nonnegative signal mean . Its maximum-likelihood estimator is , and the likelihood ratio statistic against is
This tests a specified window with a specified background, and is not either experiment's discovery significance. If the background mean is allowed to vary freely, background alone can choose and fit that count exactly. Control samples, a defensible mass-shape model, nuisance parameters and corrections for searching many masses are therefore essential to the experimental inference.
An interest-respecting reparametrization preserves the parameter of interest, or changes it only by a one-to-one function of itself. The nuisance parameter may depend on both old parameters, provided the complete transformation is smoothly invertible. Thus the level sets representing the inferential target are unchanged. Orthogonality can be sought by choosing the new nuisance coordinate along solutions of the orthogonalization equation for two statistical parameters.
Nuisance parameter 2026-10-05
A nuisance parameter is a statistical parameter required to specify the distribution of observations but not itself the target of inference. A profile likelihood maximizes over it; Bayesian model evidence integrates over it using a proper prior distribution. These operations differ and need not give the same inference. A baseline hazard in a Cox proportional-hazards model is an infinite-dimensional example.
Nuisance tangent space 2026-10-07
The nuisance tangent space is the closed linear span of score functions from statistical paths that vary the nuisance parameter while fixing the target statistical parameter. It is a closed subspace of a Hilbert space inside . Removing its component from a parametric score function leaves the information that nuisance variation cannot imitate.
Write the old nuisance parameter as with . The transformed score functions are and . Their expected cross product is . Setting it to zero gives the displayed ordinary differential equation along fixed- curves, assuming positive nuisance Fisher information. Smooth solutions with distinct initial conditions give a local interest-respecting orthogonal coordinate system.
For a parameter of interest and nuisance parameter , a partial ancillary for has a marginal law independent of , possibly depending on . This differs from a fully ancillary statistic, whose law is independent of both. For independent Poisson counts with means and , their total is Poisson with mean , while the first count conditional on the total is binomial with success probability . Conditioning on the total eliminates . A partial ancillary does not automatically eliminate a nuisance parameter; the conditional law must actually be free of that parameter.
For data , parameters and model , conditional probability gives
The likelihood describes the measurement process, the prior encodes parameter information before these data, and the Bayesian evidence normalizes the posterior. A prior is a probability density with respect to a specified measure; a uniform prior in distance is not uniform in parallax. Bayes' theorem combines conditional probabilities and does not itself select an appropriate likelihood or prior.
An astronomical application starts with a forward model. For measured fluxes with predictions and known Gaussian covariance , . Correlated calibration errors belong in or in explicit calibration nuisance parameters; assuming every filter or epoch is independent can give falsely narrow uncertainties. For a faint photon-counting source, the more appropriate likelihood can instead be , with nonnegative expected source and background counts. Background measurements constrain , which is then marginalized rather than set to an exact subtracted number. The observed background-subtracted count can be negative even though the physical expected source count is nonnegative.
For a simple flux example with and a Gaussian calibration prior , completing the square gives
This explicitly shows how independent information combines. A physical nonnegative-flux prior would truncate or replace that illustrative Gaussian prior. Photometric-redshift fitting similarly compares predicted galaxy colours with measured fluxes, then integrates over luminosity, dust and other nuisance parameters to obtain . Multimodal posteriors should be retained when different spectral features produce similar colours; a best fit alone hides that ambiguity. The same framework applies to orbit parameters, stellar temperatures, cosmological parameters and cluster SZ amplitudes.
credible intervals state posterior probability conditional on the model and prior. posterior predictive checks compare replicated observations with the data and can reveal a deficient noise or source model. Sensitivity to plausible priors is especially important for weak signals and inverse problems. For competing models, posterior odds equal prior odds times the Bayes factor ; the evidence averages fit quality over the prior volume and therefore depends on proper, physically justified prior ranges. Detection thresholds and catalogue selection must enter the sampling model when inferring a population, otherwise a carefully calculated posterior can still describe a biased sample. The practical gain is a joint uncertainty distribution with nuisance parameters propagated, not merely a single fitted number.
Work in a regular statistical model, with fixed support, differentiability under the integral, and nonsingular Fisher information. Let be the score function. The blocks are orthogonal statistical parameters at when
This is global parameter orthogonality if it holds throughout the parameter domain; otherwise it is an assertion at a specified point.
For an independent size- sample, the regular maximum-likelihood estimator satisfies
where is the information in one observation. Thus the fitted blocks are independent to first order in their limiting normal distribution. The efficient information for is generally the Schur complement ; under orthogonality it is just . Estimating the nuisance block therefore has no first-order variance cost for the interest block, compared with knowing the nuisance parameter.
This also reduces local movement of the constrained nuisance fit. If is the observed information, differentiating its nuisance score equation gives
Under regular independent sampling and orthogonality, and the centered mixed block is . Thus this derivative is locally, and an change in changes the nuisance fit by only . Neither exact finite-sample independence nor vanishing observed cross-information follows from expected-information orthogonality.
For the displayed density family, is scalar and
The mean-zero score identity gives for . But
so
This is score factorization implies parameter orthogonality. If zero belongs to the parameter domain, the conclusion extends there by continuity of the regular expected derivatives. At zero the density itself has no dependence on , so identifiability and nonsingular information for that block must be treated separately.
Partition the statistical parameter as , with of interest. For each fixed , maximize over the nuisance parameter to obtain . The profile likelihood and profile log-likelihood are
Their maximizer is the interest component of the joint maximum-likelihood estimator. At a regular nuisance maximum, and its negative Hessian matrix is the Schur complement
The profile likelihood ratio is unchanged by a one-to-one nuisance reparametrization, and transforms naturally under a one-to-one interest reparametrization. Under the regular conditions of Wilks theorem, has a limiting chi-squared distribution with dimension of degrees of freedom. Profiling is maximization, rather than integration or conditioning; need not be a normalized data density, and nuisance fitting can cause appreciable higher-order distortion.
The modified profile likelihood corrects for nuisance curvature and sample-coordinate distortion. In suitable coordinates , with an ancillary statistic, set
holding fixed when differentiating the data coordinates. Up to an interest-independent factor, the Barndorff-Nielsen modified profile likelihood is
Equivalently, implicit differentiation of the nuisance score equation gives , and
The semicolon denotes sample-coordinate differentiation, not a second parameter derivative. In orthogonal coordinates, the Cox-Reid adjusted profile likelihood uses the information adjustment alone; it agrees with this modification when , as happens below.
For the Inverse Gaussian distribution, write , and . Terms independent of both parameters can be absorbed into , giving
The nuisance score function is
It is positive below and negative above it. Thus its global maximum is
independent of . Put . Then
The nuisance observed information at this fit is
Since the constrained nuisance estimator is literally the full nuisance estimator as a function of the data, in the specified sample coordinates. Hence the modified profile likelihood for inverse Gaussian shape is
For and a nonconstant positive sample, the Cauchy-Schwarz inequality gives . The corresponding fits are and . A constant sample has and no finite profile maximum; under continuous independent sampling this event has probability zero when . In particular, the modified coefficient accounts for the fitted mean nuisance parameter.
For an interest parameter and nuisance parameter , the profile likelihood is , where maximizes the likelihood at fixed . Its maximum occurs at the joint maximum-likelihood estimate of . It preserves the optimized likelihood ratios for different interest values and is unchanged by one-to-one reparametrization of the nuisance at each fixed . It generally is neither a marginal sampling density nor a conditional likelihood, and eliminating nuisance parameters by maximization can introduce appreciable small-sample distortion. Differentiating the constrained score equation gives the curvature formula
at a joint stationary point. The corresponding expected-information expression is the efficient information for the interest parameter. Under the regular conditions of Wilks theorem, twice the profile log-likelihood drop has an asymptotic chi-squared distribution with the dimension of the interest parameter as its degrees of freedom.
For scalar nuisance, the Barndorff-Nielsen modified profile likelihood may be written
where the sample-coordinate derivative holds and the ancillary statistic fixed. For vector nuisance the last two quantities are determinants. An equivalent form is , since differentiating the constrained nuisance score gives . The semicolon denotes differentiation in sample coordinates, not another ordinary parameter derivative. This Jacobian determinant adjustment is essential to the definition.
In the present model set
The log-likelihood is . Consequently
The profile score is , and its derivative is . The expression in parentheses is the variance of the fixed design values under positive weights proportional to . Thus the profile log-likelihood is strictly concave for a nonconstant design. Its score changes from to as goes from to , giving a unique finite fit. If all coincide, only the mean is identifiable and inference on alone is not defined; that exceptional design must be excluded.
The specified ancillary coordinates give . At fixed , therefore,
Meanwhile the constrained observed information is
Substitution proves
Equivalently the sample derivative of the nuisance score is , which equals at the constrained fit and gives the same cancellation. Thus the modified profile likelihood for exponential regression is proportional to the ordinary profile likelihood. This conclusion uses the given ancillary; no proof of its ancillarity is required.
An ancillary statistic has a sampling distribution independent of the unknown parameter. It describes a realized aspect of the sample configuration without supplying a marginal likelihood for the parameter. Its observed value can nevertheless be crucial to the precision of inference from the rest of the data. In a transitive transformation model, invariance gives a common distribution under every parameter value, so maximal invariants provide natural ancillaries, such as the standardized configuration in a location-scale family.
Conditional inference uses the sampling distribution of an estimator or sufficient statistic conditional on the observed ancillary. With joint coordinates , the factorization
shows that the conditional likelihood and the joint likelihood differ only by a parameter-free data factor. Conditional reference distributions, confidence intervals and p-values can still differ markedly from unconditional ones, because they adapt to the realized configuration. For a continuous ancillary one uses a regular conditional distribution or conditional density; dividing by the zero probability of a single ancillary value is not a valid definition.
An exact example makes this adaptation visible. Let independent observations be uniform on , with . Put , , and . Specifying the minimum and maximum contributes choices of observations, while each remaining observation must lie between them. Hence
The transformation , has unit absolute Jacobian determinant. Its support is , . Integrating over gives
which is independent of , proving that the range is ancillary. Dividing the joint density by this marginal density gives
The centered conditional pivot is uniform, so
has conditional coverage at every , and hence unconditional coverage as well. A large realized range leaves little room for translating the interval and gives more precise inference. This is range-conditioned uniform location inference. The range and center are not independent: the conditional spread depends on the range, even though the range is ancillary.
Completeness can give independence in other models. Suppose is a complete sufficient statistic and is ancillary. For any bounded measurable , sufficiency makes a function independent of , while ancillarity makes independent of . Since for every , completeness implies almost surely. Taking all bounded indicator functions proves independence of and . This is the content of Basu's theorem, with the proof displaying precisely where sufficiency and completeness are used. Without completeness the conclusion fails: in the uniform location example the minimum and maximum are sufficient, and contain the nondegenerate ancillary range.
There are practical qualifications. Ancillaries need not be unique, and some models supply only approximate ancillary coordinates; one must specify the conditioning surface and the accuracy intended. Conditioning on a statistic sufficient for a nuisance parameter is a different device: it may remove that nuisance even though the statistic is not ancillary for the full parameter. Conditioning indiscriminately can discard useful information. In higher-order likelihood inference, the ancillary coordinates specify how the fitted data are varied, entering both the p* density and the sample derivatives in the modified profile likelihood. Their role is therefore substantive, rather than merely a label for an extra statistic.
A marginal statistical likelihood uses a statistic whose marginal sampling distribution depends on the parameter of interest but does not involve the nuisance parameter. It integrates out other observed coordinates. A profile likelihood instead keeps the full data and, for every fixed interest parameter, fits the nuisance parameter by maximizing the full likelihood. Profiling supplies a likelihood function but usually not a normalized sampling probability density function. Here the marginalization is of data coordinates; Bayesian model evidence averages parameters against a prior and is a separate construction.
Use shape-rate notation for the gamma distribution. The independent sums have and , so their joint probability density for is
For , the absolute Jacobian determinant is u. Therefore
Integrate u using the gamma integral . The lambda factors cancel, leaving
Thus T gives a nuisance-free marginal likelihood. Discarding only terms independent of psi, its logarithm is
The factor t in the logarithm is required, as in the original PDF.
The full log-likelihood is . Its lambda derivative vanishes uniquely at
The second derivative in lambda is negative. Substituting this maximum gives . Since , it agrees with up to a psi-independent constant. This proves the Gamma-ratio marginal and profile likelihood identity. Their common maximizer is , if an interest-parameter estimate is also wanted.
The profile likelihood for the parameter of interest replaces the nuisance parameter by its constrained best fit: , with . It is not in general a normalized sampling density. A conditional likelihood instead uses the sampling law conditional on a statistic carrying the nuisance dependence. In the canonical exponential family here, conditioning on removes because the factor involving is constant over the conditioning fibre.
Let and let the observed equal . For the cumulant-generating function of at fixed ,
The saddlepoint solves , equivalently with . This is also the constrained likelihood equation, since . Assume an interior saddle, , and the usual smooth nonlattice conditions under which has a density. Exponential tilting to makes the tilted mean. A local normal approximation gives tilted density at that mean; undoing the tilt gives the saddlepoint density approximation
The factor is for the sample mean, rather than the sum. A transform-based derivation gives the same expression by expanding to second order at its stationary point.
The joint log-likelihood is . Subtracting cancels every appearance of the original , leaving
Any change from the ambient density to a conditional density on contributes only a data-dependent Jacobian determinant. Thus, up to a constant independent of , the saddlepoint conditional likelihood adjustment is
Equivalently up to the constant , where is the nuisance observed information. The adjustment has a positive sign. The printed exponential-family form alone does not guarantee a density for : for example a continuous can have . In a lattice case one must approximate the probability mass of the sum instead; the displayed density calculation uses the regular continuous interpretation.