Maximum-likelihood estimator

An MLE maximizes sample likelihood. Regularly, $\sqrt{n}(\hat{\theta }-\theta )\Rightarrow N(0,I(\theta {)}^{-1})$; parameter-dependent support can change both rate and limit.

A growing number of nuisance parameters can prevent a maximum-likelihood estimator of a common parameter from being consistent. With two independent normal observations of mean ${\mu }_i$ per group, the profiled variance estimate ${\sum }_i(X_{1i}-X_{2i}{)}^2/(4n)$ tends to half the true variance, because each unknown mean is estimated using only two observations.

For density $r(x-\theta {)}^{r-1}$ on $[\theta ,\theta +1]$, $0<r<1$, the interior likelihood function increases without bound as $\theta$ approaches the sample minimum. The extended-likelihood estimator is that minimum. An arbitrary finite reassignment of the density at the endpoint can eliminate an attained finite maximum, so the endpoint convention must be stated.

For independent observations from the exponential distribution with rate $\theta$, the log-likelihood is $n\log \theta -\theta {\sum }_i{X}_i$, soThe central limit theorem and delta method give

A maximum-likelihood fitted value is a response mean or probability evaluated at a maximum-likelihood estimate. In a parametric mean model ${\mu }_i(\theta )$ it is ${\hat{\mu }}_i={\mu }_i(\hat{\theta })$.

For independent and identically distributed random variables $X_1,\dots ,X_n\sim N(\mu ,{\sigma }^2)$, the maximum-likelihood estimators areThe first is unbiased, while $\mathbb{E}{\hat{\sigma }}^2=(n-1){\sigma }^2/n$.

If $\hat{\theta }$ maximizes a likelihood and the target parameter is $\eta =g(\theta )$, a maximum-likelihood estimate of $\eta$ is $g(\hat{\theta })$, with the usual interpretation when $g$ is not one-to-one.

For observed data $x$, the likelihood function is the joint probability mass or density $p_{\theta }(x)$ regarded as a function of the model parameter $\theta$.

An observed-data likelihood integrates the full-data statistical probability density over unobserved values, rather than inserting arbitrary values or discarding incomplete records. When the observation pattern is random, its mechanism belongs to the joint observation model unless the conditions for an ignorable missingness mechanism permit its factor to be omitted.

A Bayesian deviance is minus twice the log-likelihood, with any chosen additive data-only constant held consistent across the models being compared. Its Bayesian posterior expectation measures average fit. In the deviance information criterion, evaluating it at a parameter's posterior mean also enters the effective complexity penalty. This likelihood-based convention differs by a data-only constant from a saturated-model exponential-family deviance when a common saturated model exists.

A conditional likelihood function conditions on an initial observation or another specified statistic. In an autoregressive model, conditioning on the first observed state avoids modeling its initial density. For Gaussian transitions, maximizing this likelihood is a regression problem; treating an autoregressive coefficient as known changes the exact distribution of an estimated mean.

For a parameter of interest $\psi$ and nuisance parameter $\lambda$, the profile likelihood isIt compares values of $\psi$ after fitting the nuisance parameter as favourably as possible at each value.

The log-likelihood is the logarithm of the likelihood as a function of the model parameter. Its maximizers are the same because the logarithm is strictly increasing.

For $X\sim \mathrm{Bin}(n,\theta )$, maximizing ${\theta }^X(1-\theta {)}^{n-X}$ gives

For independent exponential observations with rate $\theta$, the MLE is $1/\overline{X}$ and has asymptotic variance ${\theta }^2/n$.

In a regular $p$-parameter model with one-observation Fisher information $I({\theta }_0)$,