= Maximum-likelihood estimator
{title2=$\widehat\theta(X)$}
= MLE
{c}
{synonym}
= Maximum-likelihood estimate
{synonym}
A <maximum-likelihood estimator> is an <estimator> whose value maximizes the <likelihood function> for each sample where a finite maximizer exists. Before observation, $\widehat\theta(X)$ is a <random variable> (the choice of maximizer must be a <measurable function>); after observation, $\widehat\theta(x)$ is the numerical <maximum-likelihood estimate>. A formula must specify the allowed parameter set, including whether its endpoints belong to it.
For ten independent <Bernoulli trials>, write $K=\sum_{i=1}^{10}X_i$. On the closed parameter interval $[0,1]$, the <binomial proportion maximum-likelihood estimator> is $\widehat p=K/10$. Eight successes give the estimate $0.8$. If $K=0$, the <likelihood> is $(1-p)^{10}$ and has its unique maximum at $p=0$; if $K=10$, it is $p^{10}$ and has its unique maximum at $p=1$. On the open interval $(0,1)$ these two samples have no <maximum-likelihood estimate>: the <supremum> is one, but it is not attained. This is the same issue as <likelihood supremum at an excluded boundary>.
A complete continuous example uses <independent and identically distributed random variables> $X_i\sim N(\mu,1)$ with unknown <mean> $\mu\in\mathbb R$ and known <standard deviation> one. Their <likelihood function> and <log-likelihood> are
$$
L(\mu;x)=(2\pi)^{-n/2}\exp\left[-\frac12\sum_{i=1}^n(x_i-\mu)^2\right],\qquad \ell(\mu;x)=-\frac n2\log(2\pi)-\frac12\sum_{i=1}^n(x_i-\mu)^2.
$$
The <sum> of squared deviations separates as
$$
\sum_{i=1}^n(x_i-\mu)^2=\sum_{i=1}^n(x_i-\bar x)^2+n(\mu-\bar x)^2,
$$
because $\sum_i(x_i-\bar x)=0$. The first term is fixed in $\mu$, so the unique <maximum-likelihood estimator> is the <sample mean> $\widehat\mu=\overline X$. This also proves the equivalence with <ordinary least squares> for this model. With the constructed observations $[-1,0,2,3]$, the <sample mean> is one and the residual sum of squares is ten; hence
$$
\ell(\mu;x)=-2\log(2\pi)-5-2(\mu-1)^2,\qquad \widehat\mu=1.
$$
At $\mu=0$ or $2$, the <log-likelihood> is two below its peak, so the <likelihood ratio> to the fitted value is $e^{-2}\approx0.135335$. The absolute <likelihood> at the peak is $(2\pi)^{-2}e^{-5}\approx0.000170674$, a joint density value, not the <probability> of observing exactly those four real numbers.
The following figure compares that sample with a constructed sixteen-observation sample consisting of four repetitions of the displayed four values, modeled as sixteen independent draws. Both fits have <sample mean> one, but
$$
\ell_n(\mu)-\ell_n(1)=-\frac n2(\mu-1)^2,\qquad \frac{L_n(\mu)}{L_n(1)}=e^{-n(\mu-1)^2/2},\qquad -\ell_n''(\mu)=n.
$$
Thus the <observed information> is four times as large for $n=16$ as for $n=4$. The distance from the peak at relative height $e^{-2}$ decreases from one to $1/2$: the width scales as $1/\sqrt n$. This is a comparison of two stipulated samples; merely copying recorded observations would not create new independent evidence.
\Image[/gaussian-mean-likelihood.png]
{title=Known-variance normal mean: the same fitted mean with different likelihood curvature}
Even uniqueness needs checking. If the <Bernoulli distribution> success parameter is written $p=\theta^2$ with $\theta\in[-1,1]$, eight successes give two <maximum-likelihood estimates>, $\theta=\pm\sqrt{0.8}$, because both give the same <probability distribution>. The problem is a failure of <identifiability>. The special cases below show other ways an <MLE> can have <bias> or fail the usual <asymptotic normality>.
Back to article page
