If has an exponential distribution with rate , a full observation has Fisher information . Recording only for a fixed instead gives a Bernoulli distribution with parameter . The chain rule for its score function givesTo see the strict inequality, write . For the first two terms already exceed . For , the function is positive at one, its derivative is positive there, and that derivative is increasing for . Thus every positive threshold discards information. For independent observations, both information quantities multiply by the sample size. The Cramér-Rao bound and asymptotic normality of a maximum likelihood estimator explain the corresponding loss of attainable precision.
Inverse Gaussian mean estimation 2026-10-25
For an independent statistical sample from an Inverse Gaussian distribution with mean and known shape , the maximum-likelihood estimator of is the sample mean. To prove this, expand the log-likelihood function to get . Its score function is , positive below and negative above it. Taking the negative expected second derivative gives one-observation Fisher information . Thus the regular asymptotic normality of a maximum likelihood estimator yields . The same conclusion follows from the central limit theorem and the distribution's variance .
Maximum likelihood estimation Created 2026-09-24 Updated 2026-10-14
Maximum likelihood estimation is a method of statistical estimation: within a specified statistical model, choose the statistical parameter values that maximize the likelihood function of the observed data. For a parameter set and observations , its exact requirement isThe equality requires an attained, finite maximum; a supremum approached only at an excluded boundary is not enough. The resulting rule is a maximum-likelihood estimator. The likelihood function uses a probability mass function for discrete data and a probability density function for continuous data. See MIT 18.05, Reading 10b, sections 3 and 3.2.
The direction of the comparison is the interesting part: hold the observations fixed and vary the statistical parameter. Ten independent Bernoulli trials with eight successes and two failures give the maximum-likelihood estimate , as derived in log-likelihood. This chooses the member of the statistical model that assigns the greatest mass to that outcome. It does not assert an posterior probability that this parameter value is true. Bayesian statistics combines the same likelihood function with a prior distribution to obtain a posterior distribution; maximum a posteriori estimation maximizes that posterior mass or density instead.
The method turns many fitting tasks into one maximization problem. In the normal distribution model with known variance, it recovers the sample mean and ordinary least squares, while binomial regression and Poisson regression use different sampling assumptions to build their own likelihood functions. The log-likelihood turns independent contributions into a sum, exposing score equations and observed information. A peak's location gives an estimate; its shape records how much the observed data distinguish nearby parameter values.
Existence, uniqueness, bias, and statistical consistency are separate questions. Normal mean and variance maximum-likelihood estimators include a variance estimate with bias, likelihood supremum at an excluded boundary gives nonexistence, and the Neyman-Scott incidental parameter problem gives inconsistency. The usual asymptotic normality of a maximum likelihood estimator requires assumptions: a consistent estimator at an interior true parameter in a fixed finite-dimensional model with identifiability, sufficient smoothness and control of log-likelihood derivatives, and finite nonsingular Fisher information. A common parameter-independent support of a probability distribution is part of a standard sufficient framework; see Rigollet, MIT 18.650, Maximum Likelihood Estimation, slides 22-23. This is a description of the regular setting, not a complete theorem's hypotheses. The exact endpoint limit for a shifted exponential distribution shows why changing support can produce a different rate and a nonnormal limit. These properties do not follow just from maximizing a function.
In his 1922 paper in the Royal Society's Philosophical Transactions A, R. A. Fisher explicitly distinguished likelihood from probability and described choosing the parameter values with greatest likelihood. Section 6 also discusses comparing likelihoods after giving the maximum value one, the convention used in the figures here. See Fisher, On the Mathematical Foundations of Theoretical Statistics, section 6, pp. 326-328, DOI 10.1098/rsta.1922.0009.
This one-parameter normal distribution model has positive mean equal to its standard deviation. It is a curved exponential family: both the linear and quadratic natural parameters depend on the same scalar statistical parameter. It combines information about the center and spread, so the sample mean alone does not use all available Fisher information.
For a statistical sample of size , let and . Up to an additive constant the log-likelihood function is . Its score function is . When , the numerator changes from positive to negative at its unique positive root, so the global maximum-likelihood estimator isThe all-zero observation vector has zero probability and is an exceptional case with an unbounded likelihood function near zero. Since and , the expected Fisher information is . Thus regular asymptotic normality of a maximum likelihood estimator gives . The optimal asymptotic combination of a normal sample mean and standard deviation attains the same first-order precision.
Past exam of the mathematics course of the University of Cambridge 2003 iii Paper 38 4 ii Solution 2026-10-14
Here the negative binomial distribution has mean parameter and size parameter . Its support in the original PDF is all nonnegative integers; a finite endpoint appearing in the TeX aid would be incorrect. The probability generating function, obtained from the generalized binomial series, isDifferentiating at one gives and . Since the variance is , this provesThe extra positive term explains its overdispersion relative to a Poisson distribution.
For one observation, write the log-likelihood asThe mean score function and its cross derivative areTaking expectations gives . In addition,For reference, with the digamma function, the size score function isIts variance is finite and positive for finite positive . Thus negative binomial mean-size parameter orthogonality makes the expected Fisher information matrix diagonal. For an independent identically distributed sample and a regular identifiable interior true parameter, asymptotic normality of a maximum likelihood estimator givesHence the asymptotic correlation is zero. This is first-order asymptotic independence, not a claim of exact finite-sample independence. Indeed the mean score yields at an interior fit, while the size fit also depends on the observed dispersion. The argument excludes a true boundary such as the Poisson limit .
Past exam of the mathematics course of the University of Cambridge 2005 iii Paper 41 2 Solution 2026-10-14
Write for the scalar intercept denoted in the question, to distinguish it from the individual means . The log-likelihood of the independent Poisson observations isDifferentiating gives the score equations for an interior maximum-likelihood estimator:In particular, when ,The left side is a weighted covariate mean whose derivative is its weighted variance, so it is strictly increasing when the covariates are not all equal. An interior solution is unique if the observed count-weighted mean is strictly between the smallest and largest covariates. At an extreme it may require a limiting slope, and if all counts are zero the maximum occurs as , rather than at finite parameters. Constant covariates make the slope and intercept unidentifiable.
Put . The negative Hessian matrix is deterministic at given parameters and is also the Fisher information matrix:Under the usual interior, increasing-information regularity for asymptotic normality of a maximum likelihood estimator, replace the unknown means by their fitted values and read the lower-right entry:Equivalently, Poisson slope information after eliminating an intercept is , with . This explains both the positive denominator for varying covariates and the loss of slope information when their spread is small.
Past exam of the mathematics course of the University of Cambridge 2006 iii Paper 42 2 Solution Created 2026-10-03 Updated 2026-10-20
Write and for the individual and total score functions. Under the usual regularity assumptions, and , the nonsingular per-observation Fisher information matrix. The central limit theorem and weak law of large numbers giveRearranging the stipulated score expansion, including its remainder, givesSlutsky theorem therefore proves the asymptotic normality of a maximum likelihood estimator:A consistent plug-in Fisher information matrix yields the Wald statistic . Under the null hypothesis, , so a test of asymptotic size rejects when exceeds the quantile of that chi-squared distribution. A consistent observed Fisher information matrix gives the same limiting test.
With an -dimensional parameter of interest and nuisance parameter , partition accordingly. The marginal asymptotic covariance matrix of is the block of , whose inverse is the efficient informationThus the nuisance-adjusted Wald statistic is , with limiting null distribution . Using alone is generally incorrect when the nuisance parameter is estimated; it is valid when the cross-information vanishes.
For the Inverse Gaussian distribution in this problem, expand the exponent to obtain the log likelihood, up to terms independent of both parameters,Its score functions areFor each positive , the first score changes sign from positive to negative at , so this is the global maximizing mean parameter of an exponential family. Substituting it in the second score givesThe reciprocal function is strictly convex, so the denominator is positive for a nonconstant sample. The profile log likelihood is strictly concave in and tends to minus infinity at both endpoints, proving this is a maximum. Under continuous model sampling, a sample of size at least two is nonconstant almost surely. For a constant sample, including a one-observation sample, no finite joint maximum-likelihood estimator exists: the likelihood increases without bound as at .
For one observation, differentiating again and using givesThis parameter orthogonality proves that estimating does not reduce the efficient information for . The Wald statistic has the same algebraic form in the two cases:Using information evaluated at instead is another first-order equivalent convention.
The printed claim of coincidence needs a first-order interpretation. Known and unknown need not give numerically identical Wald statistics, because their maximum-likelihood estimators differ. With known, direct maximization givesFor example, observations and known give and . At , the displayed Wald convention gives and , respectively. Thus exact equality is false. Under the null hypothesis, however, , hence . Their centered, standardized estimates differ by , so their Wald statistics differ by . This is the valid asymptotic coincidence supplied by inverse Gaussian shape estimation and nuisance orthogonality.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 34 2 Solution Created 2026-10-03 Updated 2026-10-14
Put and . The integrable absolute-log envelope makes these expectations finite. Compactness and continuity of the likelihood ensure that a maximum exists, and maximizing it is equivalent almost surely to maximizing .
First identify the unique maximizer of . Let on the support of the true density. Since ,Equality requires almost surely under the true distribution and no mass outside its support, so the two probability distributions coincide. By identifiability, this happens only for . Equivalently is the Kullback-Leibler divergence from the true distribution to the candidate.
Fix and let . If this set is empty there is nothing to prove. Otherwise it is compact, so continuity and the strict maximum implyOn the event , every point of has , whereas . Hence no maximizer can belong to . The supplied uniform law of large numbers therefore proves compact-parameter consistency of maximum likelihood:
Under the usual regularity conditions, with nonsingular one-observation Fisher information matrix , the asymptotic normality of a maximum likelihood estimator iswhere and is the score function.
For the uniform model, let . For a sample in its support, the likelihood is , decreasing on its nonzero region. Thus the uniform endpoint maximum-likelihood estimator isThe all-zero sample is a probability-zero exception and may be assigned an arbitrary positive estimate. Independence gives on . For ,for this probability is zero. This proves consistency directly.
Let . Its exact cumulative distribution function isFor fixed , this converges to , so . In particular the errors are of order . Explicitly, for every fixed and sufficiently large ,ConsequentlyOne violated regularity condition is parameter-independent support: the density is supported on . Differentiation under the density integral is not valid in the usual regular-model form. Indeed the interior score equals , whose expectation is not zero; the changing endpoint contributes the omitted boundary term. Thus the uniform model is handled by its exact maximum distribution, rather than by the regular asymptotic normality theorem.
Past exam of the mathematics course of the University of Cambridge 2009 iii Paper 38 2 Solution Created 2026-10-03 Updated 2026-10-14
Work at a regular admissible parameter value with , so the score identities in the question apply. For one observation from the exponential dispersion family, the score function and second derivative areThe zero expected score immediately gives . The equality between expected negative curvature and score variance then givesThusThe admissible density parameters necessarily make this variance nonnegative; the algebra does not require an unstated sign assumption beyond the regular family and .
With independent observations and canonical parameters , the chain rule gives the vector score functionA finite interior maximum-likelihood estimator satisfies its score equation, provingDifferentiating again, using , givesThis Hessian is independent of the observations, so taking expectation gives exactly the requested Fisher information matrix. In particular, is not the inverse observation-variance matrix: the linear predictor here is the canonical parameter, not the mean.
For the two-group Poisson regression, , , and . Put , , using here for the second group's mean. There are observations, andHence the total Fisher information matrix and its inverse areThe theorem on asymptotic normality of a maximum likelihood estimator says that in a regular identifiable model with nonsingular information the estimator is approximately multivariate normal, centred at the true parameter and with covariance equal to the inverse total information. Here one can regard one observation from each group as an independent pair and apply its usual iid version to such pairs. ThereforeFor a direct check, the second score equation gives the fitted second-group mean , and the first then gives the fitted first-group mean . Thus the Poisson log-rate ratio estimate is . The delta method, using independent sample-mean variances and , gives the same variance . If a group total is zero, its finite log-mean estimator fails to exist; with fixed positive true means this event has probability tending to zero exponentially as grows.
Past exam of the mathematics course of the University of Cambridge 2010 iii Paper 32 2 1 Solution Created 2026-10-03 Updated 2026-10-14
The likelihood function for the shifted exponential distribution isIt increases strictly up to the smallest observation and is zero beyond it. Thus shifted exponential maximum likelihood givesFor , independence and the survival function of the exponential distribution give the exact order statistic lawIn particular, almost surely and . This proves statistical consistency. The exact endpoint limit for a shifted exponential distribution is stronger than an asymptotic approximation:The limit is one-sided and nonnormal; the moving support endpoint explains why a regular square-root- asymptotic normality of a maximum likelihood estimator theorem does not apply here.
Past exam of the mathematics course of the University of Cambridge 2012 iii Paper 37 3 d Solution Created 2026-10-03 Updated 2026-10-07
In the species-only model, the log odds for adult species A is the intercept, so with standard error 0.07824. This also follows directly by pooling its vials: there are 570 other-species eggs among 799 eggs eaten, giving andFor a regular maximum-likelihood estimator, asymptotic normality of a maximum likelihood estimator gives , with information . Substitution of gives . An approximate 95% Wald confidence interval isThis interval treats the selected species-only model as fixed; it does not account separately for uncertainty from choosing that model.
Past exam of the mathematics course of the University of Cambridge 2014 iii Paper 30 1 Solution Created 2026-10-03 Updated 2026-10-06
For independent identically distributed observations , the likelihood function and log-likelihood areA maximum-likelihood estimator is a measurable choice , when a maximizer exists. It need not be unique. The one-observation score function is , and the Fisher information matrix isUnder the regularity assumptions, differentiation beneath the integral gives . Differentiating again gives the second information identity. Independent observations have total Fisher information .
Assume fixed parameter dimension, an interior true parameter , a positive-definite matrix , the standard differentiability and integrability conditions, and statistical consistency of . Then the asymptotic normality of a maximum likelihood estimator isHere is the information per observation. Thus the leading covariance matrix of the estimator itself is .
To prove this, statistical consistency places the estimator in an interior ball about with probability tending to one, where its score function vanishes. Write and use the integral first-order Taylor formula for a vector map:This integral matrix is needed in a vector problem; one does not have to assert a common scalar mean-value point for every score component. PutThe regular local uniform law of large numbers, continuity of the expected Hessian matrix, and statistical consistency give . The multivariate central limit theorem givesThe limiting information matrix is nonsingular, so . Solving the Taylor identity and applying the Slutsky theorem proves the displayed normal limit, since . Boundary parameters or singular information are outside this regular theorem.
Past exam of the mathematics course of the University of Cambridge 2018 ii Paper 4 13J c Solution Created 2026-09-24 Updated 2026-10-03
Let be the parameter-contrast vector for the specified game: it has entries for , entries for , and zero elsewhere. PutUnder the stated regularity assumptions, asymptotic normality of a maximum likelihood estimator giveswhere . Since , the delta method givesEquivalently, for large ,with .
Past exam of the mathematics course of the University of Cambridge 2023 ii Paper 1 29K a Solution Created 2026-09-24 Updated 2026-09-25
Under the usual regularity conditions, consistency and asymptotic normality of a maximum likelihood estimator givewhere is the one-observation Fisher information matrix. In a scalar model this reads .
For a fixed true parameter , the usual asymptotic normality of maximum likelihood estimators givesEquivalently, for a large sample, is approximately normally distributed with mean and variance . Replacing the unknown variance by its consistent plug-in estimate gives the Wald confidence intervalHere is a standard-normal quantile. Intersect with the physical parameter range if needed. This is an interior, large-sample approximation; the degenerate estimate from does not justify a zero-width finite-sample interval.
For an interior true parameter and large , standard asymptotic normality of a maximum likelihood estimator givesEstimate its standard error by . A two-sided approximate Wald confidence interval is , intersected with the parameter range . The normal quantile is chosen so a standard normal lies between its positive and negative values with probability .
For the independent statistical sample, write the likelihood function and log-likelihood function asThe expected Fisher information in the whole statistical sample isUnder the regularity conditions that permit differentiation under the integral sign, the score function has expected value zero andFor an interior statistical parameter, assuming identifiability and , the regular asymptotic normality of a maximum likelihood estimator statesThus the large-sample normal approximation is . The inverse information is the leading sampling variance approximation; the displayed convergence specifies its precise asymptotic meaning.
Put , and . The normal distribution model givesand, up to a constant independent of , its log-likelihood function isThe score function and its zero equation areFor , this quadratic equation has exactly one positive root, giving the maximum-likelihood estimatorThe numerator of the score function is positive between its negative and positive roots and negative above its positive root. Thus the likelihood function increases up to and decreases afterwards: this is its unique global maximum on . Also, at both ends. The all-zero observation vector has probability zero; on that exceptional vector and there is no attained maximum.
The second derivative and the relevant moments areTherefore the expected Fisher information isThe support of a probability distribution is the whole real line, independent of the positive statistical parameter. The law of large numbers gives and in probability, so the displayed maximum-likelihood estimator is a consistent estimator. The regular asymptotic normality of a maximum likelihood estimator now yieldsThus the requested leading asymptotic variance on the unscaled estimator's scale is . The normal mean proportional to standard deviation model provides a reusable version of this calculation.
Write , with the natural logarithm. The likelihood function and log-likelihood function areThe score function is and the second derivative is . Thus the log-likelihood function is strictly concave, its stationary point is the unique global maximum, and the maximum-likelihood estimator isThe one-observation Fisher information is . The support of a probability distribution here is independent of , so the regular asymptotic normality of a maximum likelihood estimator givesThere is also a direct check: has an exponential distribution with rate , since for . Its mean and variance are and . Apply the central limit theorem to and the delta method to to obtain the same result.
Expanding the exponent, the log-likelihood function iswhere is independent of . Its score function and second derivative areThe first is positive below and negative above it, so the unique maximum-likelihood estimator isUsing , the one-observation Fisher information is . The support does not depend on the interior parameter , and the regular asymptotic normality of a maximum likelihood estimator givesThis is Inverse Gaussian mean estimation with known shape one. Equivalently, the Inverse Gaussian distribution has variance , so the same result follows directly from the central limit theorem applied to its sample mean.