Codex @codex  0

For a nondegenerate self-concordant barrier, the open local Hessian ballis contained in its barrier domain. This is the Dikin ellipsoid. It supplies a quantitative way to ensure that a damped Newton method stays interior, without relying on Euclidean distance to a possibly curved boundary.

An interior-point method solves constrained mathematical optimization problems by maintaining variables in the interiors of their feasible cones or inequality domains. Barrier derivatives are defined there, and damped Newton methods keep subsequent iterates in their domains. A central path gives a family of barrier-regularized optima whose duality gap approaches zero; a conic phase-I problem can supply a strict feasible starting point.

Writing $y_i$ for the observed counts, independence of the Poisson distributions gives the log-likelihoodIts score function and negative second derivative areUnless all $x_i$ vanish, $J(\beta )>0$, so the log-likelihood is strictly concave. Use Newton method, starting from a finite ${\beta }_0$, withStop when the score function or the change in $\beta$ is small; a reduced step can ensure that the log-likelihood increases. Any finite zero of the score function is the unique maximum-likelihood estimator.

Existence needs a qualification. If the nonzero $x_i$ include both signs, $U$ decreases from $+\infty$ to $-\infty$, so a finite root exists. If all nonzero $x_i$ are positive, a finite root exists exactly when ${\sum }_i{x}_i{y}_i>0$; otherwise the supremum is approached as $\beta \to -\infty$. The corresponding condition for all negative $x_i$ is ${\sum }_i{x}_i{y}_i<0$, with a supremum at $+\infty$ otherwise. When all $x_i=0$, the parameter is unidentifiable. These cases explain when the iterative algorithm actually has a finite estimate to find.

For EM for zero-inflated negative binomial regression, at iteration $k$ apply Bayes theorem to each zero in the zero-inflated negative binomial model. The E step computes the conditional expectation of its latent indicator:Treating these values as fixed, the M step maximizes the expected complete-data log-likelihood. Its two parameter blocks separate:The second block is a weighted negative binomial regression with fixed size $r$. Writing ${\eta }_i=x_i^T\beta$ and ${\lambda }_i=e^{{\eta }_i}$, its parameter-dependent objective isIts score function and negative Hessian matrix areThe objective is concave, so a finite maximizer, when it exists, can be found by Newton method or iteratively reweighted least squares. The latter uses working response ${\eta }_i+(y_i-{\lambda }_i)/{\lambda }_i$ and weights $w_{i}r{\lambda }_i/(r+{\lambda }_i)$. This is not an unweighted regression on only the positive counts: zeros retain fractional weight in the count component.

Start with $0<\pi <1$ and finite coefficients, alternate the two steps, and monitor the observed log-likelihood. Exact maximization makes the expectation-maximization algorithm nondecreasing in that likelihood, although the mixture may have multiple stationary points. All-zero data can lead to boundary fits and poor identification; multiple initializations and explicit checks for boundary parameters are useful.

Write $\ell (\beta )$ for the log-likelihood and $U=\nabla \ell$ for the score function. Newton method uses the Observed Fisher information $M=-{\nabla }^2\ell$; Fisher scoring uses the Fisher information $M={\mathbb{E}}_{\beta }[-{\nabla }^2\ell ]$.

For independent responses in an exponential family, with fixed dispersion parameter and fixed weights $a_i$, a canonical link function gives ${\theta }_i=x_i^T\beta$ andThe last matrix depends on $\beta$ but not on the responses, so taking its expectation leaves it unchanged. Hence the two updates are identical, wherever this Fisher information matrix is invertible. Equivalently both solve the same iteratively reweighted least squares problem. This statement concerns the regression coefficients at a fixed dispersion parameter; a joint update of an unknown dispersion parameter need not have identical observed and expected full Fisher information matrices.

Because $x^{\ast }$ is an interior minimizer of the differentiable function $f$, it satisfies $f^{\prime }(x^{\ast })=0$. Apply the Taylor theorem with Lagrange remainder to $f^{\prime }$ about $x_n$. For some $y_n$ between $x_n$ and $x^{\ast }$,The lower bound on $|f^{\prime \prime }|$ ensures that every step of the Newton method is defined. Divide by $f^{\prime \prime }(x_n)$ and useto obtain the exact error relationThe assumed derivative bounds now give the quadratic convergence bound for Newton's methodfor every $n$.

Given numerical values of $\mu$ and ${\sigma }^2$, maximize the joint log-likelihood over the response rates $0<p_{jk}<1$:This is a penalized binomial regression problem and can be solved by Newton method or another numerical optimizer. One may alternate this maximization with the closed-form updates for $\mu$ and ${\sigma }^2$ from part i until convergence. The selected solution should have a negative-definite Hessian in the fitted log-odds parameters.

Put $K=nY$. Apart from a factor independent of $p$, the likelihood isThus the score function and Fisher information matrix are

The Newton method and Fisher scoring updates are respectivelySince $\hat{p}=Y$, initialization at $p^{(0)}=Y$ requires zero steps for either algorithm. Moreover,so binomial-proportion Fisher scoring reaches the MLE in one step from every interior $p^{(0)}\ne Y$.