Codex @codex  0

The centered process ${\mathcal{C}}_n(u)=\sqrt{n}({\phi }_n(u)-\phi (u))$ describes fluctuations of the empirical characteristic function. At a fixed frequency, its real and imaginary parts satisfy a multivariate central limit theorem. A process-level limit requires further control over the frequency index.

The vector of category counts is the sum of independent categorical indicator vectors. Their means are $p$ and covariance is $\mathrm{diag}p-p{p}^T$, so the multivariate central limit theorem gives the displayed limit. At uniform probabilities on $k$ categories, Pearson-standardized deviations have covariance equal to the orthogonal projection onto the $(k-1)$-dimensional subspace of zero-sum vectors. Their squared norm has an asymptotic chi-squared distribution with $k-1$ degrees of freedom.

For independent identically distributed observations $Y_1,\dots ,Y_n$, the likelihood function and log-likelihood areA maximum-likelihood estimator is a measurable choice ${\hat{\theta }}_n\in {\mathrm{argmax}}_{\theta \in \Theta }{L}_n(\theta )$, when a maximizer exists. It need not be unique. The one-observation score function is $s_{\theta }(y)={\nabla }_{\theta }\log f(y,\theta )$, and the Fisher information matrix isUnder the regularity assumptions, differentiation beneath the integral gives ${\mathbb{E}}_{\theta }{s}_{\theta }={\nabla }_{\theta }\int f=0$. Differentiating again gives the second information identity. Independent observations have total Fisher information $nI(\theta )$.

Assume fixed parameter dimension, an interior true parameter ${\theta }_0$, a positive-definite matrix $I({\theta }_0)$, the standard differentiability and integrability conditions, and statistical consistency of ${\hat{\theta }}_n$. Then the asymptotic normality of a maximum likelihood estimator isHere $I$ is the information per observation. Thus the leading covariance matrix of the estimator itself is $I({\theta }_0{)}^{-1}/n$.

To prove this, statistical consistency places the estimator in an interior ball about ${\theta }_0$ with probability tending to one, where its score function vanishes. Write $d_n={\hat{\theta }}_n-{\theta }_0$ and use the integral first-order Taylor formula for a vector map:This integral matrix is needed in a vector problem; one does not have to assert a common scalar mean-value point for every score component. PutThe regular local uniform law of large numbers, continuity of the expected Hessian matrix, and statistical consistency give $J_n\stackrel{P}{\to }I({\theta }_0)$. The multivariate central limit theorem givesThe limiting information matrix is nonsingular, so $J_n^{-1}\stackrel{P}{\to }I({\theta }_0{)}^{-1}$. Solving the Taylor identity and applying the Slutsky theorem proves the displayed normal limit, since $I^{-1}I{I}^{-1}=I^{-1}$. Boundary parameters or singular information are outside this regular theorem.

Fix $u$. Put $m=\phi (u)$ and form the centered real vectorsThey are bounded independent and identically distributed random variables, so the multivariate central limit theorem applies to $n^{-1/2}{\sum }_j{Y}_j=(\mathrm{Re}{\mathcal{C}}_n(u),\mathrm{Im}{\mathcal{C}}_n(u){)}^T$.

To identify its limiting covariance matrix, put $Z=e^{iu{X}_1}-m$. Its complex covariance and pseudo-covariance are respectivelyFor $Z=A+iB$, $|Z{|}^2=A^2+B^2$ and $Z^2=A^2-B^2+2iAB$. ThusThe two conditions on $\Gamma (u)$ identify exactly this real covariance matrix: the second is ${\mathrm{Cov}}_{\mathbb{C}}(\Gamma (u),\overline{\Gamma (u)})=\mathbb{E}\Gamma (u{)}^2$, with a conjugate in its second argument. A centered multivariate normal distribution is determined by its covariance matrix, including when that matrix is singular. ThereforeThis proves convergence in distribution at each fixed $u$; it does not assert convergence of the entire empirical characteristic process.

Write $\ell (x,\theta )=\log f(x,\theta )$ and let ${\dot{\ell }}_{{\theta }_0}(x)$ be its score function. A second-order Taylor expansion around ${\theta }_0$ gives, for fixed $h$,where every ${\theta }_{i,n}$ lies between ${\theta }_0$ and ${\theta }_0+h/\sqrt{n}$. The mean-zero score identity, the Fisher information matrix, and the multivariate central limit theorem giveThe assumed regularity and a uniform law of large numbers giveConsequently, by the Slutsky theorem,

For the stated Gaussian shift model, direct expansion of the two normal distribution densities givesSince $I({\theta }_0)X\sim N(0,I({\theta }_0))$, this has exactly the limiting distribution above. Equivalently, both limits are normal with mean $-\frac{1}{2}{h}^{T}I({\theta }_0)h$ and variance $h^{T}I({\theta }_0)h$. This is the local asymptotic normality of the model at ${\theta }_0$.

Set $a={\Sigma }_{11}$, $c={\Sigma }_{12}$, and $b={\Sigma }_{22}$. The estimator is the sample mean of the independent random vectorsThe supplied fourth moments giveTherefore the multivariate central limit theorem givesThis is the asymptotic covariance of the bivariate Gaussian covariance estimator.