Codex @codex  0

Now $p_{\theta ,\eta }(x,y)=v(x)f(y-g_{\theta }(x))$. With $h_{\theta }={\partial }_{\theta }{g}_{\theta }$ and the location score $\rho (e)=-f^{\prime }(e)/f(e)$, differentiation of the log-likelihood givesTake the usual regularity conditions $h_{\theta }\in L^2(v)$ and $\rho \in L^2(f)$ so that this score function belongs to L2 space.

For the specified nuisance statistical path, differentiating $\log (1+t\gamma (e))$ at zero gives the score function $\gamma (e)$. Normalization requires $E_f\gamma =0$. There is a second constraint: the statistical path must preserve the model's zero error expected value. ThusThis constraint follows from being a path through the model, rather than from normalization alone. For a centered symmetric logistic distribution, for example, $\gamma (e)=\tanh (e)$ is bounded and has zero expected value, but $E[\epsilon \tanh (\epsilon )]>0$; it would not preserve the mean.

For any $\psi \in L^2(v)$, independence impliesAll these products are integrable by the Cauchy-Schwarz inequality, since $\epsilon \psi (X)$ is square-integrable. Thus every such error-density score function is orthogonal to every $e\psi (x)$. Covariate-density nuisance score functions $b(X)$ are also orthogonal to these functions, since $E\epsilon =0$. This is the mean-preserving error tangent space orthogonality used in the final calculation.

First use the additional admitted form of the efficient score. Write $\tilde{\ell }=e\zeta (X)$ and let ${\tau }^2=E_f{\epsilon }^2>0$. The difference $\dot{\ell }-\tilde{\ell }$ is the orthogonal projection onto the nuisance tangent space. Part (c) makes every $e\psi (X)$ orthogonal to that space. Therefore, for all $\psi \in L^2(v)$,The function in braces belongs to L2 space; choosing it as $\psi$ shows it vanishes $v$-almost everywhere. Hence the required conditional deduction isUnder the regular tail condition $sf(s)\to 0$ at both infinities, integration by parts gives $\int s{f}^{\prime }(s)ds=-1$, and the formula becomes $e{h}_{\theta }(x)/{\tau }^2$. This also confirms the sign.

The admitted product form is an extra restriction; it does not follow for every independent-error regression model. To locate the restriction precisely, suppose the error-density nuisance statistical paths preserve both normalization and mean to first order. Their closed mean-preserving error tangent space is $\{\gamma :E_f\gamma =E_f(\epsilon \gamma )=0\}$. The full nuisance tangent space is the orthogonal sum of this space and the centered functions of $X$.

For completeness, bounded functions satisfying the two constraints are dense in the error space. Truncate an arbitrary element, subtract its expected value, and then subtract a multiple of a fixed bounded centered function $q$ with $E_f(\epsilon q)\ne 0$. Such a $q$ exists by truncating $\epsilon$, since ${\tau }^2>0$. The correction coefficients tend to zero by the Cauchy-Schwarz inequality, so the corrected truncations converge in L2 space.

Let $m=E_v{h}_{\theta }(X)$. With $E_f\rho =0$ and $E_f(\epsilon \rho )=1$, the orthogonal projection of $h_{\theta }(X)\rho (\epsilon )$ onto the error nuisance space is $m\{\rho (\epsilon )-\epsilon /{\tau }^2\}$; its projection onto the covariate nuisance space is zero. Thus the general efficient score in independent-error regression isThe orthogonality of the two terms gives the displayed efficient information. For a normal distribution of the error, $\rho (e)=e/{\tau }^2$, so this reduces to the admitted formula. It also does so when $h_{\theta }$ is constant.

A concrete counterexample to the generality of the admission is $g_{\theta }(x)=\theta x$, with $X$ uniform on $[-1,1]$ and independent standard logistic distribution error. Here $m=0$ and $\rho (e)=\tanh (e/2)$, so the actual efficient score is $x\tanh (e/2)$. It cannot equal $e\zeta (x)$ because $\tanh (e/2)/e$ is not constant. This score function is already orthogonal to every nuisance score function: its factor $X$ is centered against error-only directions, and its factor $\rho (\epsilon )$ is centered against covariate-only directions. The requested formula is valid under its stated additional admission, with the general independent-error formula above explaining its limits.

For the Poisson regression, the log-likelihood isSince ${\mu }_i=e^{x_i^T\beta }$, the score function and its derivative areConsequently a finite maximum-likelihood estimator satisfiesThese nonlinear equations are normally solved by iteratively reweighted least squares. If $X$ has full column rank, the Hessian is negative definite and any finite solution is the unique maximizer. Full rank does not by itself rule out a boundary solution at infinite coefficients for every conceivable data set. The Poisson generalized linear model has dispersion one, so there is no separate scale parameter in this likelihood.

At an interior statistical parameter $\theta$, local asymptotic normality means that there are random vectors ${\Delta }_{n,\theta }$ and a symmetric nonnegative Fisher information matrix $I_{\theta }$ such that, for each fixed $h\in {\mathbb{R}}^p$,A common stronger definition requires this expansion also for $h_n\to h$. The fixed-$h$ version suffices here. A regular identifiable model usually has positive definite Fisher information; nonsingularity is unnecessary for the contiguity conclusion.

Here are sufficient differentiability conditions for a proof by Taylor's theorem. Suppose the local probability density functions have common support, ${\ell }_{\vartheta }(y)=\log f(\vartheta ,y)$ is twice continuously differentiable near $\theta$, differentiation can pass through the normalization integral twice, the score function $s_{\theta }=\nabla {\ell }_{\theta }$ has finite second moment, and the local Hessian matrix is bounded in operator norm by an integrable envelope of a function class. These hypotheses implyThe first identity differentiates $\int f_{\theta }=1$ once; the second uses ${\nabla }^2{f}_{\theta }=f_{\theta }({\nabla }^2{\ell }_{\theta }+s_{\theta }{s}_{\theta }^T)$ and differentiates twice. The central limit theorem gives ${\Delta }_{n,\theta }=n^{-1/2}{\sum }_i{s}_{\theta }(Y_i)\stackrel{d}{\to }{N}_p(0,I_{\theta })$. Taylor's integral remainder givesThe weak law of large numbers and the continuity and integrable-envelope hypotheses make the bracket converge in probability to $-I_{\theta }/2$. Indeed, the expectation of the supremum of the Hessian matrix difference over a shrinking neighborhood tends to zero by dominated convergence, and Markov inequality controls its empirical average. This proves local asymptotic normality and explains the role of both derivatives.

Write $Q_n◃P_n$ for contiguity of probability measures: for every measurable sequence $A_n$, $P_n(A_n)\to 0$ implies $Q_n(A_n)\to 0$. Mutual contiguity requires this in both directions. A useful form of Le Cam's first lemma is the following: if the Radon-Nikodym derivative $L_n$ of the absolutely continuous part of $Q_n$ relative to $P_n$ converges in distribution under $P_n$ to $L$, with $L>0$ almost surely and $\mathbb{E}L=1$, then the two sequences are mutually contiguous. This includes the usual case $L_n=d{Q}_n/d{P}_n$. The mean-one condition ensures uniform integrability of $L_n$ and vanishing mass of any singular part; positivity ensures the reverse implication.

Take $P_n=P_{\theta }^n$ and $Q_n=P_{\theta +h/\sqrt{n}}^n$. Their likelihood ratio on the support of $P_n$ satisfies, by local asymptotic normality and the continuous mapping theorem,This limit is positive and $\mathbb{E}L=e^{-v/2}{e}^{v/2}=1$, including $v=0$. The conditions of Le Cam's first lemma therefore hold, giving mutual contiguity of the product sampling laws. The products, rather than the one-observation laws, are the relevant sequences.

For any $\epsilon >0$, let $A_n=\{\Vert {\hat{\theta }}_n-\theta \Vert >\epsilon \}$. Consistency under $P_{\theta }^n$ makes $P_{\theta }^n(A_n)\to 0$, and contiguity gives $Q_n(A_n)\to 0$. Thus the estimator remains consistent for $\theta$ under each fixed local alternative. Since $\Vert \theta +h/\sqrt{n}-\theta \Vert \to 0$, it is also consistent for the moving parameter in the sense that $\Vert {\hat{\theta }}_n-(\theta +h/\sqrt{n})\Vert \to 0$ in probability under $Q_n$.

Assume the scale family has the differentiability needed for Fisher information, and the following constant is finite and positive. With ${\ell }_f(u)=\log f(u)$, the score function isPart (b) therefore giveswhere $C$ is independent of $\sigma$. The Jeffreys prior for a scale parameter is consequentlyThe reciprocal-of-a-reciprocal in the TeX is a transcription error; the PDF has ${\sigma }^{-1}$. The expected-Hessian information identity should not be applied indiscriminately to nonregular parameter-dependent supports.

For independent identically distributed observations $Y_1,\dots ,Y_n$, the likelihood function and log-likelihood areA maximum-likelihood estimator is a measurable choice ${\hat{\theta }}_n\in {\mathrm{argmax}}_{\theta \in \Theta }{L}_n(\theta )$, when a maximizer exists. It need not be unique. The one-observation score function is $s_{\theta }(y)={\nabla }_{\theta }\log f(y,\theta )$, and the Fisher information matrix isUnder the regularity assumptions, differentiation beneath the integral gives ${\mathbb{E}}_{\theta }{s}_{\theta }={\nabla }_{\theta }\int f=0$. Differentiating again gives the second information identity. Independent observations have total Fisher information $nI(\theta )$.

Assume fixed parameter dimension, an interior true parameter ${\theta }_0$, a positive-definite matrix $I({\theta }_0)$, the standard differentiability and integrability conditions, and statistical consistency of ${\hat{\theta }}_n$. Then the asymptotic normality of a maximum likelihood estimator isHere $I$ is the information per observation. Thus the leading covariance matrix of the estimator itself is $I({\theta }_0{)}^{-1}/n$.

To prove this, statistical consistency places the estimator in an interior ball about ${\theta }_0$ with probability tending to one, where its score function vanishes. Write $d_n={\hat{\theta }}_n-{\theta }_0$ and use the integral first-order Taylor formula for a vector map:This integral matrix is needed in a vector problem; one does not have to assert a common scalar mean-value point for every score component. PutThe regular local uniform law of large numbers, continuity of the expected Hessian matrix, and statistical consistency give $J_n\stackrel{P}{\to }I({\theta }_0)$. The multivariate central limit theorem givesThe limiting information matrix is nonsingular, so $J_n^{-1}\stackrel{P}{\to }I({\theta }_0{)}^{-1}$. Solving the Taylor identity and applying the Slutsky theorem proves the displayed normal limit, since $I^{-1}I{I}^{-1}=I^{-1}$. Boundary parameters or singular information are outside this regular theorem.

For an event at $t_j$ from subject $i_j$, with covariate vector $z_i$, define the Schoenfeld functionThe second term is the hazard-weighted mean covariate in the risk set just before the event. The Schoenfeld residual is this function evaluated at the fitted coefficient, $r_j=s_j(\hat{\beta })$. Calculate one residual vector per event, using every at-risk subject, including those who will subsequently be censored. There is no ordinary event residual assigned at a right censoring time.

The Cox partial likelihood score function is ${\sum }_j{s}_j(\beta )$. At the true constant coefficient in a Cox proportional-hazards model, the conditional event subject is selected with weights proportional to $e^{{\beta }^T{z}_i}$, so each Schoenfeld function has conditional mean zero. If the coefficient varies with time, that centering changes. Plot residuals against event time or a transformation of it, smooth them, and investigate departures from zero. Scaled Schoenfeld residuals account for the risk-set covariate variance and can display departures in coefficient units; score function tests based on residual-time association provide a formal check. Risk-set composition affects unscaled residual variance, and the total residual score function can be zero by fitting even when a time trend is present.

For a binomial distribution, the score function isIts squared expected value is $I(\theta )=n/[\theta (1-\theta )]$, using the binomial distribution variance $n\theta (1-\theta )$. Hence the Jeffreys prior isthe Beta distribution $\mathrm{Beta}(1/2,1/2)$. The factor $\sqrt{n}$ is independent of $\theta$ and disappears on normalization.

Writing $y_i$ for the observed counts, independence of the Poisson distributions gives the log-likelihoodIts score function and negative second derivative areUnless all $x_i$ vanish, $J(\beta )>0$, so the log-likelihood is strictly concave. Use Newton method, starting from a finite ${\beta }_0$, withStop when the score function or the change in $\beta$ is small; a reduced step can ensure that the log-likelihood increases. Any finite zero of the score function is the unique maximum-likelihood estimator.

Existence needs a qualification. If the nonzero $x_i$ include both signs, $U$ decreases from $+\infty$ to $-\infty$, so a finite root exists. If all nonzero $x_i$ are positive, a finite root exists exactly when ${\sum }_i{x}_i{y}_i>0$; otherwise the supremum is approached as $\beta \to -\infty$. The corresponding condition for all negative $x_i$ is ${\sum }_i{x}_i{y}_i<0$, with a supremum at $+\infty$ otherwise. When all $x_i=0$, the parameter is unidentifiable. These cases explain when the iterative algorithm actually has a finite estimate to find.

For EM for zero-inflated negative binomial regression, at iteration $k$ apply Bayes theorem to each zero in the zero-inflated negative binomial model. The E step computes the conditional expectation of its latent indicator:Treating these values as fixed, the M step maximizes the expected complete-data log-likelihood. Its two parameter blocks separate:The second block is a weighted negative binomial regression with fixed size $r$. Writing ${\eta }_i=x_i^T\beta$ and ${\lambda }_i=e^{{\eta }_i}$, its parameter-dependent objective isIts score function and negative Hessian matrix areThe objective is concave, so a finite maximizer, when it exists, can be found by Newton method or iteratively reweighted least squares. The latter uses working response ${\eta }_i+(y_i-{\lambda }_i)/{\lambda }_i$ and weights $w_{i}r{\lambda }_i/(r+{\lambda }_i)$. This is not an unweighted regression on only the positive counts: zeros retain fractional weight in the count component.

Start with $0<\pi <1$ and finite coefficients, alternate the two steps, and monitor the observed log-likelihood. Exact maximization makes the expectation-maximization algorithm nondecreasing in that likelihood, although the mixture may have multiple stationary points. All-zero data can lead to boundary fits and poor identification; multiple initializations and explicit checks for boundary parameters are useful.

The score function isWith common parameter-independent support and justified differentiation under the integral, ${\mathbb{E}}_{\theta }[{\partial }_{\theta }\log f(X_1,\theta )]=\int {\partial }_{\theta }f(x,\theta )dx={\partial }_{\theta }1=0$. Linearity then gives $\boxed{{\mathbb{E}}_{{\theta }_0}{S}_n({\theta }_0)=0}$. These regularity conditions exclude, for example, moving-support uniform models.