= Solution
One sufficient <maximum likelihood estimation> <statistical consistency> theorem is the following. Let the observations be <independent random variables> with a common density $f_\theta$ relative to a fixed measure, let $\Theta$ be a compact metric parameter set containing the true value $\theta_0$, and suppose that the expected <log-likelihood>
$$
m(\theta)=\mathbb E_{\theta_0}\log f_\theta(Y_1)
$$
is finite, continuous, and uniquely maximized at $\theta_0$. Assume also the <uniform law of large numbers>
$$
\sup_{\theta\in\Theta}\left|\frac1n\sum_{i=1}^n\log f_\theta(Y_i)-m(\theta)\right|\xrightarrow{\mathbb P}0,
$$
and assume that a measurable global <maximum-likelihood estimator> exists. Then \b[every such maximum-likelihood estimator converges in probability to the true parameter]. A useful sufficient condition for this <uniform law of large numbers> is almost-sure continuity of $\theta\mapsto\log f_\theta(Y_1)$ on the compact parameter set and an integrable envelope $\sup_\theta|\log f_\theta(Y_1)|\le M(Y_1)$, $\mathbb E_{\theta_0}M(Y_1)<\infty$. Under these conditions, <identifiability> supplies uniqueness through the <Kullback-Leibler divergence> identity
$$
m(\theta_0)-m(\theta)=D_{\mathrm{KL}}(f_{\theta_0}\|f_\theta)\ge0,
$$
with equality only at $\theta=\theta_0$.
To see the <statistical consistency> conclusion, fix $\varepsilon>0$. Compactness and the unique maximum give a strictly positive separation gap
$$
\delta_\varepsilon=m(\theta_0)-\sup_{d(\theta,\theta_0)\ge\varepsilon}m(\theta)>0
$$
whenever the set outside the neighbourhood is nonempty. If $\Delta_n$ denotes the supremum discrepancy in the <uniform law of large numbers>, maximization of the empirical <log-likelihood> yields
$$
0\le m(\theta_0)-m(\widehat\theta_n)\le2\Delta_n.
$$
Hence the probability of $d(\widehat\theta_n,\theta_0)\ge\varepsilon$ is at most $\mathbb P(2\Delta_n\ge\delta_\varepsilon)$, which tends to zero. This is the <argmin consistency under uniform convergence in probability> argument applied to the negative <log-likelihood>. These are sufficient conditions, not necessary ones; parameter-dependent support can require a direct argument, and independent observations need not have a common distribution in other models.
Back to article page
