Complete sufficient statistic Created 2026-10-07 Updated 2026-10-14
A sufficient statistic is complete when every integrable function of it with expectation zero for every parameter is almost surely zero for every parameter. Full regular one-parameter exponential families with natural parameter ranging over an open interval provide a standard completeness result. Completeness makes an unbiased function of a sufficient statistic unique, giving the Lehmann–Scheffé theorem.
Curved exponential family 2026-10-14
A curved exponential family, with , restricts a -parameter canonical family to a smooth rank- parameter surface, usually a nonaffine surface. For example, with has sufficient statistics and natural parameters , constrained by . Canonical concavity in the ambient parameters need not imply concavity along the curved parameter surface.
In a regular one-parameter statistical model, equality in the Cramér-Rao lower bound for an unbiased estimator implies the score function . Integrating in the parameter on a regular connected interval gives , with and . Thus efficiency for every parameter value forces an exponential family structure, under the usual common-support and differentiation assumptions.
In the positive-support exponential family , a proper natural conjugate prior with zero endpoint density values has mean parameter strictly inside the essential convex support . Updating that mean by a positive-weight average with supported observations keeps it inside . At finite parameter endpoints, prior vanishing forces , so the increased posterior power still vanishes. At positive infinity, positive base-measure mass below a point smaller than the updated mean bounds below by , giving exponential decay of the posterior kernel. Mass above the updated mean gives the analogous bound at negative infinity. Thus the updated density is proper and has the boundary condition required to integrate its score.
For known size , the negative binomial distribution is an exponential family with natural parameter and cumulant . A generalized linear model can use a logarithmic link function for its mean, even though that link is not canonical. Jointly estimating the size changes the family and variance function, requiring additional updates beyond a single fixed-family GLM fit; this is what
MASS::glm.nb does.For a finite list of real values , maximize information entropy with , , and . Lagrange multipliers giveThe constrained second variation is for any nonzero tangent perturbation, proving uniqueness of the entropy maximum whenever a feasible interior solution exists. The mean has derivative , so for nonconstant it decreases strictly with and ranges between the largest and smallest as ranges over the real line. Thus a finite exists uniquely exactly when .
At an extreme mean, strictly positive probabilities are impossible unless all coincide. If all , the mean constraint is redundant when , the maximum is uniform, and is not identifiable. This finite exponential family is an exponential tilting of the uniform distribution.
Minimal exponential family 2026-10-14
An order- exponential family is minimal if no nonzero linear combination of its canonical sufficient statistics is almost surely constant under the base measure. Equivalently, its sufficient-statistic support lies in no proper affine hyperplane. Its cumulant function then has positive-definite Hessian matrix on the interior natural domain, because that Hessian is a covariance matrix.
Natural conjugate credibility identity 2026-10-07
For a positive-support exponential family , normalization gives . The natural conjugate prior proportional to has logarithmic derivative . If its endpoint density values vanish, integration gives prior expected value . Observations update to and to . The endpoint control for a Laplace-family conjugate posterior justifies the same mean identity after updating, giving an exact credibility estimate with credibility factor .
Natural conjugate prior Created 2026-09-28 Updated 2026-10-05
For an exponential family with density proportional to , its natural conjugate prior has density proportional to . Observations update by adding sufficient statistics and update by adding sample size.
Natural exponential family 2026-10-14
A natural exponential family uses its observation vector itself as the canonical sufficient statistic. Its cumulant function is and its natural parameter space consists of parameters for which this integral is finite. On the interior, differentiating gives mean parameter of an exponential family and covariance matrix .
Past exam of the mathematics course of the University of Cambridge 2001 iii Paper 32 6 iii Solution Created 2026-10-03 Updated 2026-10-14
A finite-dimensional exponential family has common dominating measure and densityHere is the natural parameter of an exponential family, and normalization determines the cumulant function of an exponential family . A full regular family lets vary over an open natural parameter domain. A curved exponential family restricts to a lower-dimensional submanifold, while a minimal representation removes redundant affine combinations of the statistic.
Differentiating the normalization under regularity givesConsequently is convex and its Hessian is the Fisher information matrix in natural coordinates. For an independent size- sample, is a sufficient statistic: the joint density factors into times , so the conditional law given loses its parameter dependence. An interior maximum-likelihood estimator solvesmatching empirical and theoretical sufficient-statistic means. Under minimality the Hessian is positive definite, so the log-likelihood is strictly concave; existence of an interior maximizer still needs the observed statistic to be in the appropriate interior convex support.
For a full open natural family, the canonical sufficient statistic is a complete statistic: if throughout the open domain, exponential tilting about an interior point makes the Laplace transform of the integrable signed measure zero on a neighborhood. Transform uniqueness makes that measure zero, so almost surely. This requires integrability under the family. It is not an automatic property of curved restrictions. The sufficient-statistic reduction, explicit information and cumulant identities make these families especially useful for exact and higher-order inference. Normal distributions, Poisson distributions and gamma distributions supply familiar examples.
A transformation model is built differently: a group acts on the sample and parameter spaces, and its probability laws satisfyIn a transitive model all laws are transforms of a fixed reference law. The location family is generated by translations, with density . The positive-scale family has density , and the location-scale family combines them asThe Jacobian factor is essential. For independent observations its power is . The base density need not have moments, and a transformation model need not have fixed support.
An equivariant estimator respects the parameter action. With a well-defined equivariant fit valued in the transformation group, the normalized sampleis a maximal invariant. Equivariance gives ; if normalized samples agree, is obtained from by . In a transitive transformation model its law is parameter-free, so it is an ancillary statistic. For location and positive scale, centered standardized residuals provide this construction on nondegenerate samples. If the parameter action has a nontrivial stabilizer, normalization must retain its quotient action rather than pretending that a unique group element is determined.
This geometry motivates conditioning on an ancillary and using the P-star approximation for the fitted parameter. Ancillarity means parameter-free distribution; it does not by itself assert independence from the estimator. In a normal location model with known scale, the sample mean and centered residuals are independent by their joint Gaussian law and zero covariance. In an exponential scale model, the sample sum and its normalized proportions are independent, again an additional model property.
The two classes overlap but are distinct. A normal location-scale family is both an exponential family and a transformation model; an exponential mean family is both a one-dimensional natural exponential family and a scale model. A general Cauchy distribution location family is a transformation model without a fixed finite-dimensional regular exponential-family sufficient-statistic reduction. Exponential-family structure emphasizes sufficient statistics and natural-parameter convexity; transformation structure emphasizes equivariance, group orbits and ancillary reduction.
Past exam of the mathematics course of the University of Cambridge 2001 iii Paper 32 6 ii Solution Created 2026-10-03 Updated 2026-10-14
Consider a regular -parameter statistical model for which the data can locally be expressed as , where is the maximum-likelihood estimator and is a suitable exact or higher-order ancillary statistic. Write the log-likelihood as , and letbe its fitted observed information. The P-star approximation to the conditional sampling density of , given , isThe normalizing factor is independent of and is chosen so the density integrates to one over the feasible estimator coordinates. Its leading Gaussian value is , but exact normalization can change it. Both the fitted likelihood and the fitted information vary with ; the observed dataset is not held fixed while integrating over this sampling coordinate.
To see the structure, expand the likelihood difference for close to . Its leading term is minus one half of the information-weighted quadratic displacement, while supplies the local density scale. This recovers the first-order multivariate normal distribution of an efficient maximum-likelihood estimator. The formula is invariant under smooth one-to-one parameter changes: at a likelihood maximum the score vanishes, so the Hessian transforms as a quadratic form. Its square-root determinant supplies exactly the Jacobian determinant required for the estimator density to transform.
There is a direct derivation in a full regular exponential family. Let be the sufficient statistic for an independent sample, with likelihoodThe maximum-likelihood equation is , and its derivative is . Applying a multivariate saddlepoint density approximation to givesChanging variables from to multiplies by , producing the displayed P-star form. In more general models, ancillary conditioning supplies the relevant local sample coordinates; an ancillary is not an arbitrary extra statistic.
For an illustrative exact case, take an independent sample from an exponential distribution with mean , so . The fitted observed information is , andThus P-star has density shape . Normalizing giveswhich is exactly the gamma distribution sampling density of the mean. Here sample ratios provide ancillary coordinates and are independent of the mean.
In suitable regular models the unnormalized leading construction has relative error of order , and higher-order ancillary constructions with normalization can attain relative order . Such accuracy needs the stated smoothness and ancillary hypotheses; it is not a universal consequence of asymptotic normality. The exact scale-model example is likewise a special property, not a general identity. The formula is useful for refined conditional inference, score distributions, and nuisance-likelihood adjustments, including modified profile likelihood. It describes a sampling density of the estimator, rather than a posterior density of the parameter.
Past exam of the mathematics course of the University of Cambridge 2005 iii Paper 42 1 Solution Created 2026-10-03 Updated 2026-10-14
The profile likelihood for the parameter of interest replaces the nuisance parameter by its constrained best fit: , with . It is not in general a normalized sampling density. A conditional likelihood instead uses the sampling law conditional on a statistic carrying the nuisance dependence. In the canonical exponential family here, conditioning on removes because the factor involving is constant over the conditioning fibre.
Let and let the observed equal . For the cumulant-generating function of at fixed ,The saddlepoint solves , equivalently with . This is also the constrained likelihood equation, since . Assume an interior saddle, , and the usual smooth nonlattice conditions under which has a density. Exponential tilting to makes the tilted mean. A local normal approximation gives tilted density at that mean; undoing the tilt gives the saddlepoint density approximationThe factor is for the sample mean, rather than the sum. A transform-based derivation gives the same expression by expanding to second order at its stationary point.
The joint log-likelihood is . Subtracting cancels every appearance of the original , leavingAny change from the ambient density to a conditional density on contributes only a data-dependent Jacobian determinant. Thus, up to a constant independent of , the saddlepoint conditional likelihood adjustment isEquivalently up to the constant , where is the nuisance observed information. The adjustment has a positive sign. The printed exponential-family form alone does not guarantee a density for : for example a continuous can have . In a lattice case one must approximate the probability mass of the sum instead; the displayed density calculation uses the regular continuous interpretation.
Past exam of the mathematics course of the University of Cambridge 2005 iii Paper 42 3 Solution Created 2026-10-03 Updated 2026-10-14
Put , and . Apart from a data-only constant, the log-likelihood isThe natural parameters in the ambient two-parameter exponential family of normal distributions are and , restricted to the nonaffine curve , . Its parametrization has rank one. Thus this is a one-dimensional curved exponential family inside a two-dimensional full family.
The Fisher-Neyman factorization theorem makes sufficient. For two samples , the parameter-dependent part of their log density ratio is . A polynomial in is constant on only if both nonconstant coefficients vanish. The likelihood-ratio criterion for minimal sufficiency therefore proves is minimal sufficient.
Write with independent . Since , has a probability distribution independent of : it is an ancillary statistic. The denominator is nonzero almost surely. The Cauchy-Schwarz inequality gives ; on the requested positive branch, and .
The likelihood equation is . Its positive root isThere is exactly one positive root. The sign of the score function changes from positive to negative there, and at both ends of when , so it is the unique maximum-likelihood estimator. The useful identities areSubstitution yields, dropping the constant ,Differentiating twice and using at the fitted value gives the observed informationThe scalar P-star approximation uses . Consequently, with its normalizing factor explicitly retained,Here is chosen to make the density integrate to one, and is independent of . To see this without assuming an approximate normalizer, defineThe integral is finite and positive, and normalized is exactlyIndeed .
There is a direct check of this conditional scale density in a curved Gaussian family. In polar coordinates , the sample density times volume element is proportional to . Conditioning on fixes . The remaining angular factor is independent of , while . Thus the exact conditional radial density has the same normalized form. For the positive ancillary branch fixes the positive sign and the same one-dimensional radial calculation applies. In this scale model the normalized expression equals the exact conditional density, not merely its leading approximation.
For , compute its conditional distribution by one-dimensional quadrature:It is zero for ; with the conditional-density calculation just given, this normalized formula is exact for almost every admissible . Numerically, subtract the maximum log integrand before exponentiating to avoid overflow. The sampling probability still depends on the stipulated true ; replacing it by a fitted value would be a separate plug-in inference step.
Past exam of the mathematics course of the University of Cambridge 2005 ii Paper 1 5I Solution Created 2026-09-24 Updated 2026-10-14
Differentiate the normalization of the exponential family density with respect to , assuming the regularity that permits differentiation under its integral. The density derivative is . ConsequentlyDifferentiating the expectation once more gives , soThe log-likelihood derivative with respect to is . For the generalized linear model with linear predictor , the chain rule givesUsing independence to add the log-likelihood contributions proves the score functionHere includes the dispersion parameter . Defining instead a unit-dispersion variance function would require retaining an explicit in the denominator.
Past exam of the mathematics course of the University of Cambridge 2006 iii Paper 41 3 Solution Created 2026-10-03 Updated 2026-10-14
The Poisson distribution mass function isIt is an exponential dispersion family of order one withIndeed, and . A generalized linear model specifies independent random variables from a response exponential family, together with a linear predictor and a link function relating it to the response mean. The Poisson canonical link is .
For the insurance counts, let class and merit be the reference levels in a regression factor. The fitted Poisson regression assumesThe insured exposure is known; its logarithm is an offset, not a coefficient to estimate. The model has eight free mean parameters and fixes the dispersion parameter at one. It assumes additive class and merit effects on the log-rate scale, hence multiplicative effects on the rate scale without a class-by-merit interaction.
Apart from constants independent of the parameters, the log-likelihood isDifferentiating gives the Poisson regression margin-matching score equationsThus the fitted totals equal the observed totals in each class and merit category, including the reference categories by subtraction. In design matrix notation these equations are . Fisher scoring solves them iteratively; the reported three iterations describe the numerical fit, not an additional statistical result.
The regression intercept gives the baseline fitted claim rate per insured car year. The merit rate ratios, relative to merit at the same class, areThe class rate ratios, relative to class at the same merit, areEach predicted cell rate is the baseline rate times its class and merit multipliers. The output's coefficient-to-standard error ratios are asymptotic normal Wald test statistics under the fitted Poisson regression, despite the software label “t value”; there is no estimated Gaussian residual scale in this model. All nonreference effects have very large absolute ratios.
The null Poisson deviance is on nineteen statistical degrees of freedom; adding the seven factor coefficients reduces it by . But the final Poisson deviance is still on twelve statistical degrees of freedom, and the deviance residuals include values near and . These are far beyond what a well-fitting unit-dispersion model would normally produce. The additive Poisson model fits substantially better than a common rate, but remains grossly inadequate. A class-by-merit interaction, overdispersion, or statistical dependence between claims could contribute; the summary alone does not distinguish these explanations. Its very small nominal standard errors and formal Wald tests rely on the rejected model, so improved mean structure and error assumptions are needed before treating them as reliable uncertainty assessments.
Past exam of the mathematics course of the University of Cambridge 2006 iii Paper 42 4 iii Solution Created 2026-10-03 Updated 2026-10-14
A general exponential family of order has the representationThe canonical sufficient statistics are independent of the parameter, the support is fixed, and the cumulant function normalizes the probability density function. Minimality rules out a redundant affine relation among the sufficient statistics. If ranges over an open subset of , , and is a smooth embedding with derivative of rank , its image is a -dimensional parameter surface in the -dimensional natural parameter space: this is a curved exponential family. In a genuinely curved example the surface is nonaffine.
Take the normal distribution with . Its log density expands asIt is a curved exponential family with sufficient statistics and natural parameters , . The ambient full normal distribution family has , andThe curve is with , and its derivative with respect to has rank one. Thus the mean and variance are constrained by ; the two canonical parameters cannot vary independently. Concavity of an ambient canonical log likelihood need not persist along a curved parameter surface.
Past exam of the mathematics course of the University of Cambridge 2006 iii Paper 42 4 ii Solution Created 2026-10-03 Updated 2026-10-14
For , integrating the exponential tilt gives the moment-generating functionAt an interior natural parameter this is finite in a neighbourhood of , so differentiation under the integral sign is justified by the nearby exponential moments. Differentiating its logarithm once and twice at zero givesAlternatively the score function is ; its mean zero and its covariance matrix give the same exponential-family derivative identities. For any vector , . In a minimal exponential family this variance is strictly positive whenever , proving strict convexity of the cumulant function. For a general exponential family whose sufficient statistic is , these identities describe the mean and covariance matrix of , rather than necessarily those of itself.
Past exam of the mathematics course of the University of Cambridge 2006 ii Paper 3 5I Solution Created 2026-09-24 Updated 2026-10-14
In a generalized linear model, the linear predictor is , a linear combination of the covariates with unknown coefficients. The link function relates this predictor to the mean: . The canonical link function chooses the natural parameter of the exponential family, namely . Differentiating the normalization of the density shows that , so the canonical link function is on the admissible parameter interval.
For a Bernoulli distribution, the probability mass function can be written asThus and . The canonical link is the logit:
Past exam of the mathematics course of the University of Cambridge 2007 ii Paper 4 27I Solution Created 2026-09-24 Updated 2026-10-14
Assume a common support, a differentiable likelihood, and dominated differentiation under the integrals. The score function has mean zero. Its second moment is the Fisher information . If is unbiased, differentiation of gives , hence . The Cauchy-Schwarz inequality yieldsthe Cramér-Rao lower bound, assuming positive finite information.
Equality holds precisely when the centred estimator and score are linearly dependent almost surely. The covariance fixes the factor, giving . If the same unbiased estimator is efficient for every parameter on a regular interval, integrate this identity in :Exponentiation gives , an exponential family. This proves efficient unbiased estimator implies exponential family under the stated common-support regularity, rather than asserting it for nonregular support-dependent models.