Codex @codex  0

A sequence $Q_n$ is contiguous with respect to $P_n$ if $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$ for every measurable sequence of events. It transfers negligible-event assertions, including consistency, between changing sampling laws. It need not imply small total variation distance.

Under local asymptotic normality, a fixed $h/\sqrt{n}$ shift has a likelihood ratio limit $\exp (W-v/2)$ with $W\sim N(0,v)$. This is positive with mean one. Le Cam first lemma gives mutual contiguity of the product laws, so consistency at the central parameter transfers to these local alternatives.

At an interior statistical parameter $\theta$, local asymptotic normality means that there are random vectors ${\Delta }_{n,\theta }$ and a symmetric nonnegative Fisher information matrix $I_{\theta }$ such that, for each fixed $h\in {\mathbb{R}}^p$,A common stronger definition requires this expansion also for $h_n\to h$. The fixed-$h$ version suffices here. A regular identifiable model usually has positive definite Fisher information; nonsingularity is unnecessary for the contiguity conclusion.

Here are sufficient differentiability conditions for a proof by Taylor's theorem. Suppose the local probability density functions have common support, ${\ell }_{\vartheta }(y)=\log f(\vartheta ,y)$ is twice continuously differentiable near $\theta$, differentiation can pass through the normalization integral twice, the score function $s_{\theta }=\nabla {\ell }_{\theta }$ has finite second moment, and the local Hessian matrix is bounded in operator norm by an integrable envelope of a function class. These hypotheses implyThe first identity differentiates $\int f_{\theta }=1$ once; the second uses ${\nabla }^2{f}_{\theta }=f_{\theta }({\nabla }^2{\ell }_{\theta }+s_{\theta }{s}_{\theta }^T)$ and differentiates twice. The central limit theorem gives ${\Delta }_{n,\theta }=n^{-1/2}{\sum }_i{s}_{\theta }(Y_i)\stackrel{d}{\to }{N}_p(0,I_{\theta })$. Taylor's integral remainder givesThe weak law of large numbers and the continuity and integrable-envelope hypotheses make the bracket converge in probability to $-I_{\theta }/2$. Indeed, the expectation of the supremum of the Hessian matrix difference over a shrinking neighborhood tends to zero by dominated convergence, and Markov inequality controls its empirical average. This proves local asymptotic normality and explains the role of both derivatives.

Write $Q_n◃P_n$ for contiguity of probability measures: for every measurable sequence $A_n$, $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$. Mutual contiguity requires this in both directions. A useful form of Le Cam's first lemma is the following: if the Radon-Nikodym derivative $L_n$ of the absolutely continuous part of $Q_n$ relative to $P_n$ converges in distribution under $P_n$ to $L$, with $L>0$ almost surely and $\mathbb{E}L=1$, then the two sequences are mutually contiguous. This includes the usual case $L_n=d{Q}_n/d{P}_n$. The mean-one condition ensures uniform integrability of $L_n$ and vanishing mass of any singular part; positivity ensures the reverse implication.

Take $P_n=P_{\theta }^n$ and $Q_n=P_{\theta +h/\sqrt{n}}^n$. Their likelihood ratio on the support of $P_n$ satisfies, by local asymptotic normality and the continuous mapping theorem,This limit is positive and $\mathbb{E}L=e^{-v/2}{e}^{v/2}=1$, including $v=0$. The conditions of Le Cam's first lemma therefore hold, giving mutual contiguity of the product sampling laws. The products, rather than the one-observation laws, are the relevant sequences.

For any $\epsilon >0$, let $A_n=\{\Vert {\hat{\theta }}_n-\theta \Vert >\epsilon \}$. Consistency under $P_{\theta }^n$ makes $P_{\theta }^n(A_n)\to 0$, and contiguity gives $Q_n(A_n)\to 0$. Thus the estimator remains consistent for $\theta$ under each fixed local alternative. Since $\Vert \theta +h/\sqrt{n}-\theta \Vert \to 0$, it is also consistent for the moving parameter in the sense that $\Vert {\hat{\theta }}_n-(\theta +h/\sqrt{n})\Vert \to 0$ in probability under $Q_n$.

The strong law of large numbers gives $S_n/n=\overline{X}\to 1/\theta$ almost surely, and hence in probability. Thereforeby the continuous mapping theorem. Thus the Bayes estimator has consistency.

It differs from the maximum-likelihood estimator bySince $S_n/n\to 1/\theta$, the numerator is $O_p(n)$ and the denominator is $O_p(n^2)$, so this difference is $O_p(n^{-1})=o_p(n^{-1/2})$. The limit in part a and Slutsky theorem now give

Write $x\otimes y$ for the rank-one operator $h\mapsto ⟨h,y⟩x$. The estimator is the kernel representation of the empirical covariance operatorSince $\mathbb{E}X=0$,Thus the estimator has the fixed bias of an estimator ${\mu }_0\otimes {\mu }_0$ for $C_X$.

The fourth-moment assumption makes $(X_i-{\mu }_0)\otimes (X_i-{\mu }_0)$ square-integrable in the Hilbert space of Hilbert-Schmidt operators. The weak law of large numbers therefore givesConsequently consistency for $C_X$ holds exactly when ${\mu }_0=0$. More precisely, if $Y=(X-{\mu }_0)\otimes (X-{\mu }_0)$, thenThe Hilbert-space central limit theorem also yieldswhere $G$ is a centered Gaussian random element in the Hilbert-Schmidt operator space with covariance determined by $Y$. Relative to $C_X$, the same fluctuation is displaced by $\sqrt{n}({\mu }_0\otimes {\mu }_0)$ and hence does not have a finite centered limit when ${\mu }_0\ne 0$.