= Likelihood function
{title2=$L(\theta;x)$}
{wiki}
= Likelihood
{synonym}
For a fixed observed sample $x$, a <likelihood function> is
$$
L(\theta;x)=f_\theta(x),\qquad \theta\in\Theta,
$$
where $f_\theta$ is the <probability mass function> or <probability density function> of the observations under the <statistical parameter> $\theta$. All densities must be defined relative to the same parameter-independent <dominating measure>. The sample is fixed; the parameter is the function's argument. With <independent and identically distributed random variables>, the joint mass or density factors as $L(\theta;x)=\prod_{i=1}^n f_\theta(x_i)$. This definition and the distinction between sampling <probability> and parameter uncertainty are explained in https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2022/mit18_05_s22_class10-prep-b.pdf[MIT 18.05, Reading 10b, sections 3 and 3.2].
For an observed ordered sequence of ten independent <Bernoulli trials> containing eight successes and two failures, each success contributes $p$ and each failure contributes $1-p$, giving
$$
L_{\mathrm{ordered}}(p)=p^8(1-p)^2,\qquad 0\leq p\leq1.
$$
If the observation is instead only the total count $K=8$, its <binomial distribution> <probability mass function> sums over the $\binom{10}{8}=45$ possible ordered sequences:
$$
L_{\mathrm{count}}(p)=\binom{10}{8}p^8(1-p)^2=45L_{\mathrm{ordered}}(p).
$$
The raw heights differ: at $p=0.8$ they are approximately $0.0067108864$ and $0.301989888$. Their <maximum-likelihood estimates> are identical. More generally, multiplication by any finite positive constant $c(x)$ independent of $\theta$ leaves the maximizing parameter values and all <likelihood ratios> unchanged. Multiplication by a parameter-dependent factor can change the <MLE>; dropping such a factor is not justified. Relative likelihood height $L(\theta;x)/L(\widehat\theta;x)$ sets the peak to one, without making the area under the curve one.
Normalization of $f_\theta$ is over possible data values for each fixed parameter. It imposes no requirement that $\int_\Theta L(\theta;x)\,d\theta=1$. For example, the ordered <Bernoulli trials> <likelihood> above has
$$
\int_0^1p^8(1-p)^2\,dp=\frac19-\frac2{10}+\frac1{11}=\frac1{495}.
$$
Its integral is not one, and its values are not <posterior probabilities> for $p$. When the <Bayesian posterior> is defined, <Bayes' theorem> multiplies the <likelihood> by a <prior density> and normalizes that product. Choosing to normalize a <likelihood function> alone is an additional choice, such as using a <uniform prior> in those parameter coordinates, rather than part of the definition of <likelihood>.
For observations with a <probability density function>, the <probability> of an exact sample point is zero. If a scalar density is continuous at the observed value $x$, then
$$
\mathbb P_\theta\bigl(x\leq X\leq x+\varepsilon\bigr)=f_\theta(x)\varepsilon+o(\varepsilon),\qquad \varepsilon\downarrow0.
$$
Thus the density compares probabilities of small, equal observation intervals at different parameter values; it is not itself the <probability> of that exact real number. The equal-width factor is independent of the parameter. This gives visual intuition for density <likelihoods>, as in https://ocw.mit.edu/courses/18-05-introduction-to-probability-and-statistics-spring-2022/mit18_05_s22_class10-prep-b.pdf[MIT 18.05, Reading 10b, section 4]. The <normal distribution> example in <maximum-likelihood estimator> makes the comparison explicit. If only rounded intervals are recorded, their integrated probabilities give the exact observation <likelihood>; using midpoint densities is an approximation.
Back to article page
