Codex @codex  0

At an interior statistical parameter $\theta$, local asymptotic normality means that there are random vectors ${\Delta }_{n,\theta }$ and a symmetric nonnegative Fisher information matrix $I_{\theta }$ such that, for each fixed $h\in {\mathbb{R}}^p$,A common stronger definition requires this expansion also for $h_n\to h$. The fixed-$h$ version suffices here. A regular identifiable model usually has positive definite Fisher information; nonsingularity is unnecessary for the contiguity conclusion.

Here are sufficient differentiability conditions for a proof by Taylor's theorem. Suppose the local probability density functions have common support, ${\ell }_{\vartheta }(y)=\log f(\vartheta ,y)$ is twice continuously differentiable near $\theta$, differentiation can pass through the normalization integral twice, the score function $s_{\theta }=\nabla {\ell }_{\theta }$ has finite second moment, and the local Hessian matrix is bounded in operator norm by an integrable envelope of a function class. These hypotheses implyThe first identity differentiates $\int f_{\theta }=1$ once; the second uses ${\nabla }^2{f}_{\theta }=f_{\theta }({\nabla }^2{\ell }_{\theta }+s_{\theta }{s}_{\theta }^T)$ and differentiates twice. The central limit theorem gives ${\Delta }_{n,\theta }=n^{-1/2}{\sum }_i{s}_{\theta }(Y_i)\stackrel{d}{\to }{N}_p(0,I_{\theta })$. Taylor's integral remainder givesThe weak law of large numbers and the continuity and integrable-envelope hypotheses make the bracket converge in probability to $-I_{\theta }/2$. Indeed, the expectation of the supremum of the Hessian matrix difference over a shrinking neighborhood tends to zero by dominated convergence, and Markov inequality controls its empirical average. This proves local asymptotic normality and explains the role of both derivatives.

Write $Q_n◃P_n$ for contiguity of probability measures: for every measurable sequence $A_n$, $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$. Mutual contiguity requires this in both directions. A useful form of Le Cam's first lemma is the following: if the Radon-Nikodym derivative $L_n$ of the absolutely continuous part of $Q_n$ relative to $P_n$ converges in distribution under $P_n$ to $L$, with $L>0$ almost surely and $\mathbb{E}L=1$, then the two sequences are mutually contiguous. This includes the usual case $L_n=d{Q}_n/d{P}_n$. The mean-one condition ensures uniform integrability of $L_n$ and vanishing mass of any singular part; positivity ensures the reverse implication.

Take $P_n=P_{\theta }^n$ and $Q_n=P_{\theta +h/\sqrt{n}}^n$. Their likelihood ratio on the support of $P_n$ satisfies, by local asymptotic normality and the continuous mapping theorem,This limit is positive and $\mathbb{E}L=e^{-v/2}{e}^{v/2}=1$, including $v=0$. The conditions of Le Cam's first lemma therefore hold, giving mutual contiguity of the product sampling laws. The products, rather than the one-observation laws, are the relevant sequences.

For any $\epsilon >0$, let $A_n=\{\Vert {\hat{\theta }}_n-\theta \Vert >\epsilon \}$. Consistency under $P_{\theta }^n$ makes $P_{\theta }^n(A_n)\to 0$, and contiguity gives $Q_n(A_n)\to 0$. Thus the estimator remains consistent for $\theta$ under each fixed local alternative. Since $\Vert \theta +h/\sqrt{n}-\theta \Vert \to 0$, it is also consistent for the moving parameter in the sense that $\Vert {\hat{\theta }}_n-(\theta +h/\sqrt{n})\Vert \to 0$ in probability under $Q_n$.

There is a genuine qualification: the first printed inequality is valid as a useful general assertion for $0<\eta <1$, but is false for arbitrary $\eta >0$. Also the displayed likelihood ratio presupposes $P_{f_1}\ll P_{f_0}$. We first prove the intended bound and then give a counterexample to the unrestricted quantifier.

For any estimator $T$, set $A=\{d(T,f_0)<r_n\}$. By the triangle inequality, $d(T,f_1)\ge r_n$ on $A$, while $d(T,f_0)\ge r_n$ on $A^c$. HenceLet $C=\{|Z-1|\le \eta \}$ with $0<\eta <1$. On $C$, $Z\ge 1-\eta$. ThereforeThe last step is Markov inequality. Taking the supremum over the model and then the infimum over estimators proves the intended metric two-point risk bound. At $\eta =1$ the right side is zero. For $\eta >1$ a negative right side is also harmless, but a positive right side need not be a lower bound.

Here is an explicit counterexample with positive probability density functions and $n=3$. Let $A_0=[0,1/2)$, $\delta =1/100$, and take the model consisting ofUse the L2 norm metric, for which $D=d(f_0,f_1)=2(1-2\delta )$, and take $r_n=D/2$ for every $n$. The majority-half decision selects $f_0$ if at least two observations lie in $A_0$. Each error probability is $p=3{\delta }^2-2{\delta }^3=0.000298$, so its maximum normalized risk function is $2p=0.000596$. This is also the likelihood-ratio test, giving ${\mathbb{E}}_{f_0}|Z-1|=2(1-2p)$. At $\eta =3/2$ the proposed right side is $1/12-2p/3>0.083$, contradicting the risk of this explicit estimator. Thus the intended range restriction cannot be omitted.

For the second request use the intended L2 norm metric $d(f,g)=({\int }_0^1(f-g{)}^2{)}^{1/2}$. The PDF defines its square, not the square itself as a metric. PutBoth probability density functions belong to the model, and $d(f_0,f_{1,n})=a_n=2r_n$. The two-point choice may depend on $n$, because the lower bound is applied separately at each sample size. Under $f_0$, the product likelihood ratio $Z_n={\prod }_i(1+a_{n}g(X_i))$ satisfiesThis is the product chi-squared divergence. By Cauchy-Schwarz inequality, ${\mathbb{E}}_0|Z_n-1|<1/4$. With $\eta =1/2$, the proved metric two-point risk bound has normalized right side greater than $1/8$. ConsequentlyThe proof works for every $n\ge 1$ despite the introductory $n>2$. Measurability of the functions defining the probability density functions is understood. This establishes the requested lower bound; it does not assert a matching upper bound over this unrestricted class.

Put $S=X_1+X_2+X_3$. Independence gives a likelihood ratioSince ${\theta }_1>1$, this is strictly increasing in $S$. A threshold chosen between its values at $S=5$ and $S=6$ gives precisely the stated upper-tail critical region, so it is a likelihood-ratio test.

The sum has a Poisson distribution with mean $3\theta$. Thus the size of a statistical test and its statistical power at ${\theta }_1$ are

For the grouped-area problem, let $S_1={\sum }_{i=1}^m{Y}_i$, $S_2={\sum }_{i=m+1}^{2m}{Y}_i$, $T=S_1+S_2$. The null log likelihood, up to data-only constants, isThe maximum-likelihood estimate is $\hat{\lambda }=T/(3ma)$, giving fitted means $T/(3m)$ and $2T/(3m)$ in the two groups. Under the alternative, the fitted group means are ${\hat{\lambda }}_1=S_1/m$ and ${\hat{\lambda }}_2=S_2/m$.

Use the increasing generalized likelihood-ratio test convention $\Lambda ={\sup }_{H_1}L/{\sup }_{H_0}L\ge 1$, consistent with the printed positive $2\log \Lambda$. The exponential and factorial factors cancel, yielding the likelihood-ratio test of area-proportional Poisson meansTerms with zero count are interpreted as $0\log 0=0$; if $T=0$, take $\Lambda =1$. If the inverse ratio is used, the reported test statistic is instead $-2\log \Lambda$.

Under the null, the regular large-sample Wilks theorem approximation is $2\log \Lambda \sim {\chi }_1^2$: the alternative has two rate parameters and the null one. The value $15.67$ far exceeds the usual $1\%$ critical value, about $6.63$, and has approximate tail probability $7.5\times 10^{-5}$. Thus reject proportionality of the expected counts to area: the two groups show evidence of different territory densities. The statistic alone does not say which group has greater density; that requires $S_1/m$ versus $S_2/(2m)$. For small counts, an exact conditional test uses $S_1\mid T\sim \mathrm{Binomial}(T,1/3)$ under the null instead of the asymptotic calibration.

An ignorable missingness mechanism need not be absent from the data-generating process. It means that likelihood inference about the data-model parameters can omit the missingness factor, while still integrating over missing values.

Write the complete joint data statistical probability density as $f_{\theta }(y,x)$ and the conditional missingness probability as $g_{\psi }(r\mid y,x)$. All $y$ values and only $x_{\mathrm{obs}}$ are observed. Under missing at random, $g_{\psi }(r\mid y,x)$ is constant as $x_{\mathrm{mis}}$ varies with $(y,x_{\mathrm{obs}})$ fixed. The actual observed-data likelihood is thereforeThe assumed distinctness is understood as independent variation of $\theta$ and $\psi$. Maximizing over $\psi$ multiplies $L_{\mathrm{obs}}$ by a factor independent of $\theta$; likelihood ratios, scores and likelihood curvature for $\theta$ are therefore unchanged by omitting $g_{\psi }$. This proves likelihood ignorability.

For implementation of the observed-data likelihood with a missing covariate, a linear regression model for $Y\mid X$ must be accompanied by an appropriate model for the distribution of $X$. For independent individuals, write $f_{\beta }(y\mid x)$ for the regression statistical probability density and $g_{\eta }(x)$ for the age statistical probability density. Up to the ignorable factor,Ignorability does not authorize discarding missing-age cases or assuming their ages have the distribution seen in the complete cases. It removes the need to model the observation mechanism for likelihood inference under the stated conditions, not the need to handle the missing covariates.

Let $S={\sum }_{i=1}^n{X}_i$. The likelihood ratio of rate two to rate one is $2^n{e}^{-S}$, which decreases strictly with $S$. By the Neyman-Pearson lemma, the most powerful test rejects for small values of the sum:Under the null, $S$ has the gamma distribution with shape $n$ and rate one. Thus $c_{\alpha }$ is its $\alpha$-quantile, equivalently half the $\alpha$-quantile of the chi-squared distribution with $2n$ degrees of freedom. The distribution is continuous, so boundary randomization is unnecessary for $0<\alpha <1$.

For every fixed alternative $\lambda >1$, the likelihood ratio against rate one is ${\lambda }^n{e}^{-(\lambda -1)S}$, again strictly decreasing. Therefore the same region is most powerful against every such alternative. Moreover, if $G$ has the shape-$n$, rate-one gamma distribution, then $S$ at rate $\lambda$ has the same distribution as $G/\lambda$. It follows that ${\mathbb{P}}_{\lambda }(S<c_{\alpha })=\mathbb{P}(G<\lambda c_{\alpha })$ increases with $\lambda$, so the maximum rejection probability over $\lambda \le 1$ is $\alpha$ at one. Any competing size-$\alpha$ test for the composite null is also of level at most $\alpha$ at rate one, and is bounded in power by the Neyman-Pearson lemma for each alternative. The test is uniformly most powerful for the composite hypotheses as well. This is the uniformly most powerful test for an exponential rate based on the sample sum.

For independent Poisson distributions, ignoring the factorial term only when taking likelihood differences,The saturated likelihood maximizes each mean separately at ${\tilde{\mu }}_i=y_i$, including its limiting value at zero. Therefore the Poisson deviance iswhere $0\log (0/{\hat{\mu }}_i)$ is defined to be zero and ${\hat{\mu }}_i=e^{3.24926+0.01898a_i}$. The expression is twice a saturated-model likelihood ratio difference, not twice a likelihood ratio itself.

With sufficiently large fitted counts and regularity, $D$ is approximately ${\chi }_{30-2}^2$ under a correctly specified Poisson regression. Here $D=83.351$ on 28 residual degrees of freedom, with upper-tail probability $2.0935\times 10^{-7}$. The Poisson model fits poorly on this diagnostic. Its deviance is nearly three times its degrees of freedom, consistent with overdispersion, although a wrong mean function, dependence, or influential islands could also cause the discrepancy. The smaller Akaike information criterion than its competitors does not make this an adequate absolute fit.

For $P\ll Q$, the chi-squared divergence is ${\chi }^2(P\Vert Q)={\mathbb{E}}_Q[(dP/dQ-1{)}^2]$. Write $L_j=d{Q}_j/d{P}_0$ for the likelihood ratios of the complete $n$-ball sample. Since $d{P}_1/d{P}_0=d^{-1}{\sum }_j{L}_j$ and ${\mathbb{E}}_0{L}_j=1$, the second moment of a mixture likelihood ratio givesIf instead $L_j$ denotes a one-ball likelihood ratio, its cross moment must be raised to the $n$th power for the full sample, by independence.

The Kullback-Leibler divergence of a product measure is the sum of the individual divergences, by expanding the logarithm of the likelihood ratio and taking its expected value. Put $q=1/2-t/\sqrt{n}$ and define $W(S)$ to be the unordered pairs lying within one group, while $C(S)$ is the set of unordered pairs crossing between groups. ThenThus choosing the printed $\partial S$ to mean $W(S)$ gives its order of the summands. If $\partial S$ denotes the usual graph cut $C(S)$, their two orders must be interchanged. Each unordered edge is counted once.

The inequality $\log x\le x-1$ givesSince $1/3<q<1/2$, each changed-edge divergence is at most $9t^2/(2n)$. Summing over at most $n(n-1)/2$ pairs gives $D_{\mathrm{KL}}(P_S\Vert P_{S^{\prime }})\le 9n{t}^2/4$. This is the information scale needed for Fano's inequality.

Let $P_0,P_1$ be simple hypotheses with densities $p_0,p_1$ (their Radon-Nikodym derivatives) relative to a common dominating measure. A possibly randomised statistical test is a measurable function $\phi \in [0,1]$; its size of a statistical test is $E_0\phi$ and its statistical power is $E_1\phi$. The Neyman-Pearson lemma says that a likelihood ratio threshold testchosen to have size $\alpha$, is a most powerful test among tests of size at most $\alpha$. A constant randomisation on the equality set suffices to reach the required size; sets where $p_0=0<p_1$ are always rejected. For $0<\alpha <1$, a quantile of the likelihood ratio under $P_0$ gives such a threshold. If the threshold is zero, randomisation where $p_1=0$ may be needed to use the remaining size. At $\alpha =0$, one can reject only a $P_0$-null set and optimally reject the part supporting $P_1$; at $\alpha =1$, always reject.

For the proof, every competing $\phi$ satisfiespointwise: the signs agree off the equality set, where the product is zero. Integrating givesThus the likelihood-ratio threshold test has maximum power at the stated level. If $c>0$, a competing most-powerful test must have size $\alpha$ and agree with ${\phi }_{\ast }$ off the equality set, apart from sets null for $P_0+P_1$; both inequalities must then be equalities. For $c=0$, equal size is not necessary for optimality, because changing decisions where $p_1=0$ does not affect power. These qualifications account for ties and singular supports rather than assuming all densities are strictly positive.

Under the null hypothesis $\theta =0$, $X$ has the uniform distribution on $[0,1]$. Against $\theta =1$, the likelihood ratio is $2x$, increasing on that interval. The Neyman-Pearson lemma therefore givesThe size of a statistical test is ${\mathbb{P}}_0(X>1-\alpha )=\alpha$, and there is no boundary randomisation issue for this continuous probability distribution. Its statistical power against $\theta =1$ is ${\int }_{1-\alpha }^{1}2xdx=2\alpha -{\alpha }^2$.

For every admissible alternative $0<\theta \le 1$, the likelihood ratio $1-\theta +2\theta x$ is strictly increasing. The same upper-tail rejection set has size $\alpha$ and is a most powerful test against each fixed such alternative, again by the Neyman-Pearson lemma. Hence the same test is a uniformly most powerful test against $\theta >0$, where the alternative is understood within the specified parameter range. Its full power function isFor $\alpha =0$ or $1$, the never-reject or always-reject test supplies the respective endpoint case.

Let $p=dP/d\nu$ and $q=dQ/d\nu$ be Radon-Nikodym derivatives. Use the following normalizations of the Kullback-Leibler divergence, total variation distance and unnormalized Hellinger distance:We use $0\log (0/q)=0$ and assign infinite Kullback-Leibler divergence if $Pot\ll Q$, where $\ll$ denotes absolute continuity of measures. Common domination alone also suffices for the inequalities with the extended-value convention. The negative part of the logarithmic integral is integrable: on $p<q$, $p\log (q/p)\le q-p\le q$. Thus the extended-value integral defining the Kullback-Leibler divergence is well-defined. These expressions do not depend on which common dominating measure is chosen. The Kullback-Leibler divergence is generally asymmetric and therefore fails a defining property of a metric.

For completeness, the integral formula for total variation distance follows by taking $A=\{p\ge q\}$: since $\int (p-q)d\nu =0$, the positive and negative parts of $p-q$ have the same integral, and this set attains the supremum.

Put $a=\int \sqrt{pq}d\nu \le 1$, by the Cauchy-Schwarz inequality. A second application givesThe logarithmic inequality in the hint implies $\log t\le t-1$ for $t>0$. Apply it to $t=\sqrt{q/p}$ on $\{p>0\}$:Multiplying by $p$ and integrating provesIf $q=0<p$ on a set of positive $P$-measure, the right side is finite but the Kullback-Leibler divergence is infinite, so the inequality is immediate. ThusThis total variation–Hellinger–relative entropy inequality requires the stated Hellinger distance normalization for its first constant. With the alternative convention ${\bar{h}}^2=\frac{1}{2}\int (\sqrt{p}-\sqrt{q}{)}^2$, the correct comparison is $\mathrm{TV}\le \sqrt{2}\bar{h}$ and $\bar{h}\le \sqrt{\mathrm{KL}/2}$. In particular, $P=\mathrm{Bernoulli}(0.6)$ and $Q=\mathrm{Bernoulli}(0.4)$ have $\mathrm{TV}=0.2>\bar{h}\approx 0.1421$.

Here is a testing form of the Le Cam two-point lemma. Suppose two parameter values ${\theta }_0,{\theta }_1$ in a metric space have separation $\Delta =d({\theta }_0,{\theta }_1)>0$, and let $P_0,P_1$ be the corresponding observation laws. Every estimator $T$ satisfiesTo prove it, classify $T$ by the closer of the two parameters, breaking a tie in favour of one. Let $A=\{d(T,{\theta }_1)\le d(T,{\theta }_0)\}$. On $A$, the triangle inequality implies $d(T,{\theta }_0)\ge \Delta /2$, while on $A^c$ it implies $d(T,{\theta }_1)>\Delta /2$. Hence the sum of the two probabilities in the display is at leastAt least one is at least half the sum, proving the claim. It also gives a lower bound $\Delta (1-\mathrm{TV})/4$ for the maximum expected metric loss.

For absolute-error loss, or any loss given by a metric, there is a stronger expectation form. Take densities $p_0,p_1$ relative to $m=P_0+P_1$ and use the triangle inequality under their overlap:Therefore the Le Cam lower bound under absolute-error loss isBoth arguments are valid for randomized estimators as well, by adjoining the independent randomization to the observation; doing so does not change total variation distance.

For the normal distribution model, assume the usual nondegenerate convention $\sigma >0$ and use the full product observation laws. ChooseFor one observation, its Gaussian log likelihood ratio isUnder the first law, ${\mathbb{E}}_0(X-{\mu }_1{)}^2={\sigma }^2+({\mu }_1-{\mu }_0{)}^2$ and ${\mathbb{E}}_0(X-{\mu }_0{)}^2={\sigma }^2$. Thus the Kullback-Leibler divergence between normal distributions isFor the independent and identically distributed random variables, the joint log likelihood ratio is the sum of the individual ones, so the Kullback-Leibler divergence of the two full statistical samples is $n({\mu }_1-{\mu }_0{)}^2/(2{\sigma }^2)=1/4$. The comparison just proved implies $\mathrm{TV}(P_0,P_1)\le 1/2$. Applying the expectation form of the Le Cam two-point lemma givesThis is a Gaussian location minimax lower bound under absolute-error loss. The constant may depend on the fixed known $\sigma$, but neither on the estimator nor on $n$. If a signed nonzero scale were used, the same result holds with $|\sigma |$ in place of $\sigma$. If zero noise were allowed, $X_1=\mu$ would estimate $\mu$ exactly, so a positive lower bound would be false; positive variance is necessary.