= Maximum likelihood estimation
{wiki}
<Maximum likelihood estimation> is a method of <statistical estimation>: within a specified <statistical model>, choose the <statistical parameter> values that maximize the <likelihood function> of the observed data. For a parameter set $\Theta$ and observations $x$, its exact requirement is
$$
\widehat\theta(x)\in\Theta,\qquad L(\widehat\theta(x);x)=\sup_{\theta\in\Theta}L(\theta;x).
$$
The equality requires an attained, finite <maximum>; a <supremum> approached only at an excluded boundary is not enough. The resulting rule is a <maximum-likelihood estimator>. The <likelihood function> uses a <probability mass function> for discrete data and a <probability density function> for continuous data. See https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2022/mit18_05_s22_class10-prep-b.pdf[MIT 18.05, Reading 10b, sections 3 and 3.2].
The direction of the comparison is the interesting part: hold the observations fixed and vary the <statistical parameter>. Ten independent <Bernoulli trials> with eight successes and two failures give the <maximum-likelihood estimate> $\widehat p=0.8$, as derived in <log-likelihood>. This chooses the member of the <statistical model> that assigns the greatest mass to that outcome. It does not assert an $80\%$ <posterior probability> that this parameter value is true. <Bayesian statistics> combines the same <likelihood function> with a <prior distribution> to obtain a <posterior distribution>; <maximum a posteriori estimation> maximizes that posterior mass or density instead.
The method turns many fitting tasks into one <maximization problem>. In the <normal distribution> model with known <variance>, it recovers the <sample mean> and <ordinary least squares>, while <binomial regression> and <Poisson regression> use different sampling assumptions to build their own <likelihood functions>. The <log-likelihood> turns independent contributions into a <sum>, exposing <score equations> and <observed information>. A peak's location gives an estimate; its shape records how much the observed data distinguish nearby parameter values.
Existence, uniqueness, <bias>, and <statistical consistency> are separate questions. <Normal mean and variance maximum-likelihood estimators> include a variance estimate with <bias>, <likelihood supremum at an excluded boundary> gives nonexistence, and the <Neyman-Scott incidental parameter problem> gives inconsistency. The usual <asymptotic normality of a maximum likelihood estimator> requires assumptions: a <consistent estimator> at an interior true parameter in a fixed finite-dimensional model with <identifiability>, sufficient smoothness and control of <log-likelihood> <derivatives>, and finite nonsingular <Fisher information>. A common parameter-independent <support of a probability distribution> is part of a standard sufficient framework; see https://ocw.mit.edu/courses/18-650-statistics-for-applications-fall-2016/36624c2682b346b07cab3da416b752a7_MIT18_650F16_Maximum_LE.pdf[Rigollet, MIT 18.650, Maximum Likelihood Estimation, slides 22-23]. This is a description of the regular setting, not a complete theorem's hypotheses. The <exact endpoint limit for a shifted exponential distribution> shows why changing support can produce a different rate and a nonnormal limit. These properties do not follow just from maximizing a function.
In his 1922 paper in the Royal Society's Philosophical Transactions A, R. A. Fisher explicitly distinguished <likelihood> from <probability> and described choosing the parameter values with greatest <likelihood>. Section 6 also discusses comparing <likelihoods> after giving the maximum value one, the convention used in the figures here. See https://errorstatistics.com/wp-content/uploads/2021/04/fisher-1922.pdf[Fisher, On the Mathematical Foundations of Theoretical Statistics, section 6, pp. 326-328, DOI 10.1098/rsta.1922.0009].
Back to article page
