Binomial proportion maximum-likelihood estimator Created 2026-09-24 Updated 2026-11-05
For a binomial count , , with , the maximum-likelihood estimator is . For , differentiating gives a unique interior maximum. If or , the maximum is respectively at zero or one. Excluding those endpoints from the parameter space means these samples have only a supremum, not an attained maximum. The estimate is unbiased and has variance , by the binomial distribution moments.
Distinct parameters Created 2026-10-03 Updated 2026-11-05
Outcome and missingness parameters are distinct when their joint parameter space is the Cartesian product of their separate parameter spaces, so variation-independent choices can be made for the two models.
Equivariant estimator Created 2026-10-14 Updated 2026-11-05
An estimator respects the group action if the displayed identity holds. If it takes values in the transformation group itself, normalization is a maximal invariant: equivariance proves invariance, and equality of normalized samples reconstructs the transformation between them. For estimates in a homogeneous parameter space with a nontrivial stabilizer, normalization is defined only modulo that stabilizer; quotienting the remaining action is essential.
Estimating Bernoulli variance Created 2026-10-14 Updated 2026-11-05
For independent Bernoulli observations with sum , the extended maximum-likelihood estimator is . Its unbiased improvement is , obtained by conditioning on . The Rao-Blackwell theorem gives strict variance reduction for . With a uniform prior on and squared-error loss for , the Bayes estimator is . For parameter space strictly , samples with or have only a boundary likelihood supremum, not an attained maximum.
Finite-sample bias in a negative multinomial probability estimator Created 2026-10-14 Updated 2026-11-05
For independent one-stop negative multinomial distribution observations, the natural estimates are and . Their sum is with strictly concave , so Jensen's inequality makes its expectation strictly below . Conditional category proportions split this bias between both estimates. With zero observed counts of a category, that estimate lies at zero; in a parameter space requiring strict positivity the likelihood supremum is not attained.
Information matrix of an experimental design Created 2026-10-14 Updated 2026-11-05
The normalized information matrix for a homoscedastic linear regression model is the average outer product of its regressor vector. It is positive-definite precisely when the supported regressors span the parameter space. The full Fisher information matrix under normal errors is .
Maximum-likelihood covariance estimator for centered Gaussian data Created 2026-09-24 Updated 2026-11-05
For independent Gaussian random vectors with known zero mean and positive-definite covariance matrix , the candidate maximum-likelihood covariance estimator isWhen is positive definite, the Gaussian log likelihood, up to a constant, is . Writing the precision matrix as gives the concave objective , whose derivative vanishes uniquely at . This proves the displayed estimator. It is unbiased because the mean is known: .
If the sample vectors do not span the whole -dimensional space, is singular. Shrinking a candidate covariance eigenvalue to zero along an unobserved direction makes the likelihood unbounded, so there is no maximum within the positive-definite parameter space. In particular, cannot yield a positive-definite maximum-likelihood estimate. For a positive-definite Gaussian population and , the vectors span the space almost surely. When the mean is instead estimated from the same data, centering changes the rank condition and the unbiased covariance uses denominator rather than .
Negative binomial information in capture-recapture sampling Created 2026-10-31 Updated 2026-11-05
Suppose a population has individuals, tagged, and captures are made independently with immediate replacement. Stop each run at the th tagged capture, with known positive integer , and let count untagged captures before stopping. Then has a failure-count negative binomial distribution with tagged-capture probability and mass .
For independent runs and , the log-likelihood function apart from constants is . Treating as continuous in givesIndeed, yields the second-derivative Fisher information. For an interior true , the usual continuous-parameter asymptotic variance is . If , the likelihood is maximized at on the closed parameter space.
For actual unknown population size , the constraint restricts to a discrete set. The relaxed estimate need not be an integer; evaluate the likelihood at the admissible integers on either side of it rather than claiming the continuous optimum is an attained integer-population maximum-likelihood estimator. A confidence interval for , with , transforms to and then is intersected with the allowed integer population sizes. Immediate replacement is essential; sampling without replacement gives a different likelihood.
Nonparametric maximum-likelihood estimator Created 2026-09-24 Updated 2026-11-05
A nonparametric maximum-likelihood estimator maximizes a likelihood over a nonparametric family of distributions or functions rather than over a fixed finite-dimensional parameter space.
Parameter space Created 2026-10-31 Updated 2026-11-05
The parameter space of a statistical model is the set of allowed statistical parameter values indexing its probability distributions. For an exponential distribution parametrized by its mean, ; for a Bernoulli distribution, . Intersecting a confidence interval with the known parameter space cannot remove the true statistical parameter and therefore preserves its coverage probability.
Partial likelihood-ratio test Created 2026-09-24 Updated 2026-11-05
For a Cox proportional-hazards model, let denote the logarithm of the Cox partial likelihood. To compare a full parameter space with a nested null model imposing independent smooth restrictions, maximize the same partial likelihood under each model and formNesting makes . Under the null hypothesis, a correctly specified regular model, independent censoring conditional on covariates, fixed parameter dimension, an interior true parameter and nonsingular limiting information, . The degrees of freedom count independent restrictions, not the number of observed failures. For example, testing that two identifiable regression coefficients both vanish gives .
The calibration follows from the local quadratic expansion of : the unrestricted maximizer removes the score component in the constrained directions, whose standardized squared length has a chi-squared distribution. It requires enough event information for this approximation. A variance parameter constrained to be nonnegative and tested at zero, or a parameter unidentified under the null model, violates the regular interior assumptions; an ordinary reference cannot be presumed. Model comparisons must also use the same observations and the same event-tie convention.
Here the exponential distribution is parametrized by its mean, so for and . Its log-likelihood and maximum-likelihood estimate areFor one observation, . Since , its Fisher information is . Consequently the estimated standard error is , givingThis is a large-sample confidence interval, not an exact one. If its lower endpoint is negative, intersecting with the parameter space removes impossible values without excluding any possible true statistical parameter.
Statistical model Created 2026-09-24 Updated 2026-11-05
A statistical model is a specified family of probability distributions for the observations. A statistical parameter often indexes it as , where is the parameter space. The choice states which patterns of variation are considered possible and supplies a basis for statistical inference; it does not guarantee that any member exactly describes the data-generating mechanism.
For example, suppose binary observations are independent and identically distributed with a common unknown success probability . The Bernoulli distribution model has and likelihood functionWith and seven successes, this becomes and is maximized at . The binary observation type alone does not supply independence or a common : those are additional model assumptions. A regression model can instead let the response law depend on observed covariates, while a mixed model represents a specified grouping structure through random effects.
Transformation model Created 2026-10-14 Updated 2026-11-05
A group action on the sample space induces a compatible action on the parameter space, preserving the family of probability laws. In a transitive model every law arises by transforming a fixed reference law. A location-scale model is generated by , , with parameter action .