Codex @codex  0

Put $Q_n(\theta )=n^{-1}{\sum }_{i=1}^n\log f(Y_i;\theta )$ and $Q(\theta )={\mathbb{E}}_{{\theta }_0}\log f(Y_1;\theta )$. The integrable absolute-log envelope makes these expectations finite. Compactness and continuity of the likelihood ensure that a maximum exists, and maximizing it is equivalent almost surely to maximizing $Q_n$.

First identify the unique maximizer of $Q$. Let $r(y)=f(y;\theta )/f(y;{\theta }_0)$ on the support of the true density. Since $\log r\le r-1$,Equality requires $r=1$ almost surely under the true distribution and no mass outside its support, so the two probability distributions coincide. By identifiability, this happens only for $\theta ={\theta }_0$. Equivalently $Q({\theta }_0)-Q(\theta )$ is the Kullback-Leibler divergence from the true distribution to the candidate.

Fix $\epsilon >0$ and let $K_{\epsilon }=\{\theta \in \Theta :\Vert \theta -{\theta }_0\Vert \ge \epsilon \}$. If this set is empty there is nothing to prove. Otherwise it is compact, so continuity and the strict maximum implyOn the event ${\sup }_{\Theta }|Q_n-Q|<{\delta }_{\epsilon }/3$, every point of $K_{\epsilon }$ has $Q_n(\theta )<Q({\theta }_0)-2{\delta }_{\epsilon }/3$, whereas $Q_n({\theta }_0)>Q({\theta }_0)-{\delta }_{\epsilon }/3$. Hence no maximizer can belong to $K_{\epsilon }$. The supplied uniform law of large numbers therefore proves compact-parameter consistency of maximum likelihood:

Under the usual regularity conditions, with nonsingular one-observation Fisher information matrix $I({\theta }_0)$, the asymptotic normality of a maximum likelihood estimator iswhere $I({\theta }_0)={\mathbb{E}}_{{\theta }_0}[S_{{\theta }_0}(Y_1)S_{{\theta }_0}(Y_1{)}^T]$ and $S_{\theta }={\nabla }_{\theta }\log f(\cdot ;\theta )$ is the score function.

For the uniform model, let $M_n={\max }_i{Y}_i$. For a sample in its support, the likelihood is ${\theta }^{-n}{1}_{\{\theta \ge M_n\}}$, decreasing on its nonzero region. Thus the uniform endpoint maximum-likelihood estimator isThe all-zero sample is a probability-zero exception and may be assigned an arbitrary positive estimate. Independence gives ${\mathbb{P}}_{\theta }(M_n\le y)=(y/\theta {)}^n$ on $[0,\theta ]$. For $0<\epsilon <\theta$,for $\epsilon \ge \theta$ this probability is zero. This proves consistency directly.

Let $T_n=n(\theta -M_n)/\theta$. Its exact cumulative distribution function isFor fixed $t\ge 0$, this converges to $1-e^{-t}$, so $T_n\stackrel{d}{\to }\mathrm{Exp}(1)$. In particular the errors are of order $1/n$. Explicitly, for every fixed $\eta >0$ and sufficiently large $n$,ConsequentlyOne violated regularity condition is parameter-independent support: the density is supported on $[0,\theta ]$. Differentiation under the density integral is not valid in the usual regular-model form. Indeed the interior score equals $-1/\theta$, whose expectation is not zero; the changing endpoint contributes the omitted boundary term. Thus the uniform model is handled by its exact maximum distribution, rather than by the regular asymptotic normality theorem.