For the mean parameter , the log-likelihood is . Its derivative is , positive below and negative above it. Therefore is the unique maximum-likelihood estimator.
The exponential distribution has mean and variance . The central limit theorem gives the pivot
Let for fixed . For , the event is exactly
Both denominators are then positive, so the algebra neither reverses the ordering nor changes the parameter domain. Taking probabilities and using continuity of the limiting normal distribution proves the exponential-mean confidence interval by pivot inversion has coverage probability tending to . This is inversion of the pivot with the true mean in its denominator; it is not the ordinary symmetric Wald interval centered at . For a small sample with , the displayed finite positive upper endpoint is not valid, although the large-sample claim is unaffected.
Use the Probabilists' Hermite polynomials, rather than the differently scaled physicists' convention:
Thus , , and . The cumulant-generating function of an exponential variable of mean is near zero, so its th cumulant is . Hence the standardized third and fourth cumulants are and ; the fourth is a cumulant, not the fourth raw central moment.
Write for the cumulative distribution function of the pivot in part (i). Its Edgeworth expansion, valid here because the exponential law is smooth, nonlattice and has exponential moments, is
at the fixed endpoints . The coverage is exactly . Since and are even, the terms cancel; the two odd Hermite terms double. Consequently the symmetric Edgeworth coverage cancellation gives
In particular its coverage probability error is . This is a fixed-confidence-level expansion; it is not a uniform statement for normal cutoffs increasing with .
A conditional likelihood is the conditional density of the observed data given a chosen statistic, viewed as a function of the parameter. Conditioning can eliminate a nuisance parameter if the statistic is sufficient for it within the relevant family. The conditioning statistic need not be ancillary; conditioning can also discard some information about the parameter of interest.
For the canonical family, let and . Normalization of the density gives the cumulant-generating function of one nuisance statistic:
For the sample mean , it is . Differentiation gives and .
Assume an interior saddle with positive curvature and the usual smooth nonlattice density conditions. Define by
This is the constrained maximum-likelihood estimator of with held fixed. The exponential tilt parameter for one summand is . Tilting puts the sample mean at , where its tilted density has leading normal value . Undoing the tilt gives the saddlepoint density approximation
This is the density of the mean, not the sum; their factors differ by .
The sample log-likelihood is . Conditional on , the factor involving is constant on the conditioning surface, so the conditional distribution is free of . Subtracting the saddlepoint log-density from the joint log-likelihood now cancels all the original nuisance terms:
The final term, and any conditioning-coordinate factor, are data-only constants. Thus the saddlepoint conditional likelihood adjustment is
The absolute-value notation agrees with the question; positive variance makes it unnecessary in this regular setting. The sign of this correction is positive, because a nuisance-statistic density with inverse-square-root curvature has been divided out. A lattice-valued statistic requires the corresponding mass approximation, and a boundary or zero-variance saddle is outside the stated regular density calculation.
Let be a suitable ancillary statistic and assume the sample can locally be written in coordinates , where is the scalar maximum-likelihood estimator. Ancillarity means the sampling law of does not involve . For each hypothetical sample coordinate , write for its log-likelihood, and
The P-star approximation to the conditional density with respect to is
Here is chosen so the integral over the admissible domain is one; its leading regular approximation is . One must vary the data coordinate inside the fitted information and likelihood, not evaluate those factors once at the observed estimate and treat them as constant. The expression combines the likelihood-ratio shape with the local information-volume factor. A quadratic likelihood gives the usual normal approximation, while the full shape retains skewness and other higher-order effects.
The approximation is invariant under smooth one-to-one parameter transformations: at the fitted point the transformed observed information acquires the square of the coordinate Jacobian, so its square root supplies exactly the density Jacobian. Under regular smooth fixed-dimensional likelihood and ancillary conditions it is a higher-order conditional approximation, typically with relative error of order , and in suitable refined settings of order . These are not universal error guarantees for singular, boundary or arbitrary ancillary constructions. In important transformation models the normalized formula is exact. The normalizing factor may depend on ; treating it as a universal constant can itself lose higher-order accuracy.
For a concrete check, in the exponential-mean model of Question 1 the likelihood difference is and . The resulting P-star density for an exponential mean normalizes to
which is exactly the density of . This example demonstrates the importance of both the information factor and normalization; it does not prove exactness for every regular model.
For the score density, fix an evaluation parameter , and define . On any locally monotone branch, let . The ancillary-conditioned score density is obtained by an ordinary change of variable:
If several branches give the same score, sum their contributions. The sampling parameter and the score-evaluation parameter have been distinguished; for the score at the true parameter take . The denominator is a data-coordinate derivative, not automatically .
For the conditional distribution function of the estimate, integrate the normalized density:
Both integrals use the actual support and observed ancillary value. Numerical quadrature gives tail probabilities or confidence limits by inversion. Conditional inference should not silently average over the ancillary, because it is the conditional distribution that this construction is designed to approximate.
A functional statistic has the form , where is the empirical distribution and is a statistical functional on a class of distributions. It separates the population quantity from its sample estimate. Fisher consistency for a model means for every model parameter; it does not alone establish consistency of the empirical estimate without continuity and probabilistic conditions.
The influence function measures infinitesimal contamination at a point:
when this derivative exists. A bounded influence function limits first-order sensitivity to an arbitrarily remote observation.
An M-estimator is a minimizer of an empirical loss, or a chosen consistent solution of an estimating equation . Its population functional solves . Put and , assumed finite and nonzero. Differentiating the contaminated population equation gives
since the unperturbed expected score is zero. Hence
Under differentiability, consistency and the uniform local laws needed for expansion of the empirical equation,
The central limit theorem now yields asymptotic normality with sandwich variance of an M-estimator
Thus the leading variance of itself is the displayed variance divided by . For a vector parameter, replace by the derivative matrix and use .
In a location model with known scale , choose . For a symmetric reference distribution, an odd score is Fisher consistent. With and positive , the formulas become
The Gaussian likelihood score gives the efficient sample mean under normality, but its unbounded influence is vulnerable to large outliers. The Huber score is linear centrally and bounded in the tails, balancing normal-model efficiency against contamination sensitivity. Its denominator is for a continuous law. The sign score gives the median and needs a nonsmooth derivation using the density at the median, rather than incorrectly setting its ordinary derivative to zero. Redescending scores can reject very extreme observations, but may have multiple roots and require a consistent root-selection rule.
Thus appropriate choices seek Fisher consistency, a nonzero stable derivative, finite variance, reasonable reference-model efficiency and bounded influence. Robust scale estimation may be needed for the tuning constant; joint location-scale estimation uses the matrix influence calculation rather than treating an estimated scale as known. Bounded influence describes local robustness and is not by itself a global breakdown point theorem.
For , the probability integral transform makes iid uniform on . Continuity of is enough; strict increase is unnecessary. Their extreme order statistics have joint density
choose the observations giving the two extremes and put the remaining between them. Transform to and integrate from zero to . The uniformized sample range therefore has
Equivalently . For , the range is identically zero instead.
The exact failure probability is
Thus the exact integer sample size is the least making this quantity at most . For with , it tends to . This function decreases strictly from one to zero on positive , because its derivative is . For there is therefore exactly one positive solution of
This gives the extreme-order-statistic tolerance interval sample-size rule
When is small, , so a useful first refinement is . This is an approximation, not an exact integer guarantee; use the preceding exact failure probability to check rounding and very stringent tail requirements. The interval here encloses population content, rather than estimating a fixed distribution parameter.
For the Wilcoxon signed-rank test, assume independent observations from a continuous distribution symmetric about the specified . Form , rank the absolute differences , and add the ranks of positive differences:
Under the continuous null there are no zeros or tied absolute values with positive probability. Symmetry makes the signs independent fair signs conditional on the magnitudes, so the exact null distribution is that of for independent . In particular
The coefficients of this polynomial give exact tail probabilities, or they can be computed by recursively adding a rank with probability one half. A two-sided test rejects for unusually small , equivalently for extreme on either side of its center. Choose symmetric tail cutoffs with total null probability at most ; discrete boundary randomization can attain size exactly . An exact conservative two-sided p-value is . For a positive-location alternative use the upper tail, and for a negative-location alternative use the lower tail.
For large samples, centering and scaling by the displayed mean and standard deviation gives an asymptotic standard normal statistic. This follows from the independent weighted Bernoulli representation, because the largest squared weight is negligible relative to the sum of squared weights. A continuity correction improves the normal tail approximation. With observed zero differences, omit them in the usual convention; tied absolute differences receive average ranks, and the conditional null variance is then one quarter of the sum of squared assigned ranks, not automatically the no-tie formula.
Symmetry is essential: having median alone need not make the signs independent of the magnitudes, and so does not justify these exact critical values. The test is particularly useful for location shifts in symmetric families. It is a valid test of the stated symmetry null, but is not an omnibus procedure guaranteed to detect every asymmetric alternative.
A Bartlett correction improves calibration of a regular likelihood-ratio test statistic , whose null limit is for tested restrictions. If
the corrected statistic is . Its null mean is and it is compared with the same limiting chi-squared distribution. The coefficient is model-specific, may involve nuisance parameters, and can be calculated from likelihood cumulants or a suitable null simulation; substitution must respect the asymptotic regime.
Under the regular higher-order likelihood expansions and identities that establish Bartlett correctability, this rescaling removes the leading distributional error and can give chi-square calibration error. Matching one moment by itself does not prove that assertion for an arbitrary statistic, and nonregular boundary or singular models need different treatment.
For illustration, testing a normal variance with unknown mean gives with . The gamma logarithmic moment yields
Thus the Bartlett correction for a normal variance with unknown mean uses , not the correction for the different known-mean model. This example also distinguishes the relative coefficient from the full divisor .
For parameter of interest and nuisance parameter , the profile likelihood has log form , where is the constrained nuisance maximum. Profiling optimizes over the nuisance dimension but does not account for its local volume or conditioning geometry. The modified profile likelihood corrects this loss.
Use ancillary sample coordinates , and let be the nuisance block of observed information. Holding fixed, put . The Barndorff-Nielsen modified profile likelihood is, up to data-only factors,
This combines a nuisance-information determinant and a data-coordinate Jacobian determinant. Implicit differentiation of gives , so equivalently
The semicolon denotes differentiation in the varying sample coordinate, rather than a parameter derivative. This form also explains the positive adjustment in Question 2: in that canonical conditional problem the mixed sample derivative is data-only, leaving the positive half-log curvature. The adjustment sign cannot be copied between unrelated constructions without tracking their Jacobians.
In an appropriate orthogonal nuisance formulation, the Cox-Reid adjusted profile likelihood uses . It is a useful simpler adjustment, but discarding is not universally legitimate. For normal data with interest and nuisance , the constrained and full nuisance fits both equal , so and . Thus
recovering the residual degrees of freedom and fitted variance . These adjustments target nuisance-related bias and higher-order likelihood inference; they are approximations to conditional or marginal constructions, not an assertion that profiling has become an exact full sampling density.
The Laplace approximation evaluates an integral dominated by an interior maximum. Suppose has a unique interior maximizer , negative-definite Hessian there, and sufficient smoothness and tail control. Put . Taylor expansion under gives a Gaussian leading integrand and
The leading factor follows from the multivariate Gaussian integral. Odd cubic terms integrate to zero over the limiting symmetric domain; fourth-order terms and squared cubic terms contribute to the first correction, assuming enough derivatives. If , a different leading term may be needed.
For a likelihood integral with prior density , a common version is , because observed information is of order . This is useful for posterior normalization, marginal likelihood and approximating integrated nuisance effects. A prior or integration measure is part of that construction; integrating nuisance parameters is different from merely profiling them. Boundary maxima, degenerate curvature, multiple modes or uncontrolled tails invalidate the simple one-mode formula.
In a transformation model, a group acts on data and parameters compatibly: implies . A statistic is invariant if , and is a maximal invariant if equality of its values also implies the samples lie on the same group orbit. A maximal invariant records all information unchanged by the transformations; measurable invariant procedures therefore factor through it, subject to the ordinary orbit-space measurability conditions.
An equivariant estimator satisfies for the induced action on the quantity estimated. For the positive affine location-scale group, with , this means and .
For a nonconstant ordered sample, take and . Then
is a maximal invariant: it is unchanged by the positive affine group, and if then , explicitly recovering the transformation. Both mean and standard deviation are equivariant. If the whole model varies only by location and positive scale, the normalized sample's distribution is independent of those two parameters, giving an ancillary statistic. It may still retain information about additional shape parameters.
Equivariance yields a coherent transformation rule and, under an invariant loss, can simplify risk comparison to a reference parameter. It does not on its own establish optimality. Normalization by a parameter-space estimator with a nontrivial stabilizer requires quotienting that remaining action; otherwise an apparently standardized statistic need not be maximal.
Partition a regular parameter into interest and nuisance . Parameter orthogonality means the cross-block of the expected Fisher information matrix vanishes:
It is expected-information orthogonality, not a demand that every observed cross derivative be zero. To first order, the asymptotic normal distribution of the joint maximum-likelihood estimator has independent interest and nuisance blocks. Its efficient interest information simplifies from the Schur complement to .
For scalar interest, new nuisance coordinates can locally be chosen by the local orthogonal nuisance reparametrization
Indeed the new interest score is , and its expected product with the transformed nuisance score is zero by this equation. Smoothness, nonsingular nuisance information and suitable initial coordinates provide the local ordinary-differential-equation construction. Simultaneous orthogonalization for several interest coordinates may face integrability constraints.
For normal location and variance , the cross information is zero: the location score is proportional to the centered observation and the variance score to its squared value minus its expectation, whose product has zero expectation by normal symmetry. This is a useful orthogonal parameterization. In general orthogonality reduces nuisance sensitivity and motivates adjusted profile constructions, but it implies neither exact estimator independence nor an exact nuisance-free conditional likelihood.
A one-sample U-statistic of fixed degree has a symmetric U-statistic kernel and averages it over all distinct subsets of an iid sample:
It is unbiased for . With finite second moment, define the first Hoeffding projection . Orthogonality of the higher projections gives
If , the central limit theorem therefore proves the U-statistic central limit theorem
Equivalently the influence function of a U-statistic is , obtained by differentiating its population functional in each distribution argument. If , the root- limit degenerates and the second or higher projection determines a different, often nonnormal, limit.
For example, gives the unbiased sample variance with denominator . Its first projection is , so the influence function is . When the fourth moment is finite, the leading variance of the sample variance is . This example connects unbiased symmetric estimation to the functional influence calculation.
Several finite-sample counterparts of the influence function should be distinguished. The empirical influence function substitutes the empirical distribution: . For a smooth M-estimator it is
It requires a well-defined derivative and nonsingular denominator. An unsmoothed empirical quantile functional, for example, need not admit the population-style density derivative.
The actual one-observation sensitivity curve is a finite difference:
It records contamination of size rather than an infinitesimal perturbation. For a smooth functional it approaches the population influence curve as the sample grows. A deletion counterpart uses , giving
The jackknife resampling pseudovalue is therefore approximately . The jackknife variance estimate corresponds asymptotically to for centered empirical influences.
In linear regression, Cook's distance measures the fitted-value effect of deleting a case. For coefficients, residual , leverage and variance estimate , its usual expression is . It illustrates that a case can be influential through both a large residual and high leverage. Finite perturbations and nonlinear refitting can differ materially from a first-order influence approximation; these diagnostics should not be conflated.
A curved exponential family restricts a higher-dimensional full canonical family to a lower-dimensional smooth parameter surface:
The derivative matrix has rank . A genuinely curved surface is not merely an affine subfamily under a choice of natural coordinates. With iid observations the ambient sufficient statistic is , and the parameter score and information are
The second-derivative terms from have zero expectation because the centered sufficient statistic has mean zero. Under regularity, the constrained MLE is still first-order asymptotically normal with this information. Curvature affects higher-order bias, likelihood geometry and efficiency; concavity of the ambient likelihood need not survive along the nonlinear parameter surface.
For example, with has and natural parameters , , constrained by . It has one free parameter but generally still needs both and as sufficient statistics. To see why, a likelihood ratio between two samples is constant in only if their statistic differences satisfy for all positive , forcing both differences to zero. Lower parameter dimension therefore does not generally reduce the minimal sufficient statistic to that dimension.

Articles by others on the same topic (0)

There are currently no matching articles.