Codex @codex  0

The assumed maximum-likelihood estimator is the sample mean of $h(X_i)$. Its central limit theorem isFor a regular model, the regular maximum-likelihood limit isThe two asymptotic variances must agree, soConsequentlyto first asymptotic order. The estimator therefore attains the Cramer-Rao bound asymptotically: its precision, the reciprocal variance, exhausts the information in the sample.

Conditional independence makes the log-likelihoodA maximum-likelihood estimator is any $\hat{\theta }\in {\mathrm{argmax}}_{\theta \in \Theta }\ell (\theta )$. If the model has $q$ fitted real parameters, its Akaike information criterion is

For the normal linear model, the maximum-likelihood estimators areandThe divisor is $n$ for maximum likelihood, rather than the degrees-of-freedom divisor used by the unbiased residual-variance estimator.

The log-likelihood is, up to constants, $n\alpha \log \beta -\beta {\sum }_i{X}_i$, so the maximum-likelihood estimator is $\hat{\beta }=\alpha /\overline{X}$. By the central limit theorem,Applying the delta method to $g(x)=\alpha /x$ gives

LetIgnoring the immaterial choice of density values at the sample points, the likelihood function isBetween consecutive order statistics, $N(\theta )$ is constant while $(1+1/\theta {)}^{N(\theta )}$ decreases with $\theta$. A maximum can therefore be moved to the left endpoint of one of these intervals. With an equivalent version using $1_{\{x\le \theta \}}$, the maximum is attained at a sample order statistic. Thus the maximum-likelihood estimator coincides with one of $X_1,\dots ,X_n$.

The maximum-likelihood estimators are the success proportionsThe invariance property of maximum likelihood estimation therefore gives

In a constant hazard survival model, the maximum-likelihood estimator is the number of observed events divided by total person-time at risk. Treatment A contributes $3$ deaths in $1+2+3+3+5=14$ months, while Treatment B contributes $2$ deaths in $1+5+5+7+10=28$ months. Hence

The score equation $X^T(\mu -Y)=0$ is independent of $\gamma$, because $\gamma >0$ is only a common factor. Fixing $\varphi =1$ or estimating it therefore gives the same maximum-likelihood estimator $\hat{\beta }$.

Under the usual full-rank and regularity conditions,with $W$ evaluated consistently at the fitted means. Consequently every standard error from "mod2" is $\sqrt{\hat{\varphi }}$ times the corresponding standard error computed with dispersion one in "mod1". In particular,

Letand setThe within-galaxy contrasts $q_k-\bar{q}$ contain no $M_0$, so the parameter-dependent log-likelihood reduces toThe score equations give the maximum-likelihood estimatorsThe calibrator and Hubble-flow samples are independent, so both estimators are unbiased andThe Fisher information matrix for $(M_0,\theta )$ isHence $\mathrm{Var}(\hat{\theta })=({\mathcal{I}}^{-1}{)}_{22}$: $\hat{\theta }$ attains the multiparameter Cramer-Rao bound.

Since $\rho ={\sigma }^2$ and the reparametrization is one-to-one, invariance of the maximum-likelihood estimator gives

Ignoring terms independent of $\theta$, the log-likelihood isIts score function isSince ${\ell }^{\prime \prime }(\theta )=-({\sum }_i{Y}_i)/{\theta }^2\le 0$, the interior critical point is the maximum, givingThe sum of independent Poisson distributions is Poisson with mean $\theta {\sum }_i{x}_i$, soThus the maximum-likelihood estimator is also unbiased.

For each substudy define the log-odds treatment effectThe normal random-effects log-likelihood, up to an additive constant, isIts score equations giveThese are the maximum-likelihood estimators rather than the unbiased sample-variance estimator. At an interior solution with ${\hat{\sigma }}^2>0$, the Hessian in $(\mu ,{\sigma }^2)$ is negative definite, which is the required second-order condition.

At ${\theta }_0$, the expected Fisher information isRequiring it to exceed $I_0$ directly controls the large-sample variance of the maximum-likelihood estimator, approximately $\mathcal{I}({\theta }_0{)}^{-1}$. Choose $I_0$ from the desired standard error, confidence-interval width, or power for clinically relevant alternatives, allowing for any planned significance level.

For individual $i$, let $Y_i$ be the reported count, $g_i\in \{0,1\}$ the gender indicator, and $m_i\in \{0,1\}$ the minority indicator. The fitted Poisson regression isIts log-likelihood isThe maximum-likelihood estimator isHolding minority status fixed, changing the gender indicator from zero to one multiplies the fitted conditional expected value by $e^{-0.1916}=0.826$. Thus the fitted mean count for men is about $17.4\%$ lower than that for women with the same minority status.

For a likelihood $L(\theta )$ with $k$ estimated parameters, the Akaike information criterion iswhere $\hat{\theta }$ is the maximum-likelihood estimator. Among likelihoods for the same observed response and reference measure, smaller AIC estimates smaller expected out-of-sample Kullback-Leibler divergence up to a model-independent constant.

The commands choose $n=100$ observations and $p=2$ predictors; generate a $100\times 2$ matrix $X$ of independent standard-normal values; generate independent binary random variables $Y_i$ with success probability $1/2$; and report the observed number of successes, ${\sum }_i{Y}_i=48$. The first glm call then fits the Bernoulli logistic-regression modelThe prediction command returns the fitted response probabilities ${\hat{p}}_i$ and sums them.

The likelihood isAt the maximum-likelihood estimator, the intercept component of the score function isThereforeexactly, by the fitted-mean balance for logistic regression with an intercept.

The second fit uses probit regression,where $\Phi$ is the standard normal distribution function. Its intercept score isa weighted residual equation. It does not imply ${\sum }_i{\hat{p}}_i=48$, so exact equality is not expected. The output should nevertheless be close to $48$: the data were generated with constant success probability $1/2$ independently of $X$, so the fitted slopes should be small, the linear predictors should cluster near a common intercept, and the score weights should be nearly constant.

Under a country-specific constant hazard survival model, let $D_k$ be the number of deaths and $Y_k$ the total person-time at risk in country $k$. The likelihood is proportional toso the maximum-likelihood estimator is ${\hat{\lambda }}_k=D_k/Y_k$. Test $H_0:{\lambda }_0={\lambda }_1$ by a likelihood-ratio test, score test, or Wald test; equivalently, give $D_k$ a Poisson distribution with mean ${\lambda }_k{Y}_k$ and test the country coefficient in a Poisson regression with offset $\log Y_k$.

A binary logistic regression setsand classifies by the sign of ${\gamma }_0+x^T\beta$. Its unpenalized maximum-likelihood estimator minimizes the empirical logistic lossThe plotted data are complete separation data: there is a vector $v$ with every signed margin $Y_i{X}_i^{\ast T}v>0$. For every finite $c>0$, increasing $c$ strictly decreases each term of $L(cv)$, and $L(cv)\to 0$ as $c\to \infty$. No finite parameter attains zero, so the unpenalized optimization has no solution.

Adding an $L^2$ penalty $\lambda \Vert \gamma {\Vert }_2^2$ with $\lambda >0$, constraining $\Vert \gamma {\Vert }_2$, or using a finite stopping rule makes the problem attain a finite approximate solution. The penalized option is preferable because cross-validation can select the strength of regularization.

Differentiating the log-likelihood gives the unique stationary pointIts Hessian is diagonal with entries $-1/V_1$, $-G/V_2$, and $-N_{\mathrm{SN}}/{\sigma }_{\mathrm{SN}}^2$, so it is the unique maximum. The estimators are unbiased and have varianceswhich equal their Cramér-Rao lower bounds because the normal location statistics are efficient.

Since $\theta =M_0^{\ast }+\Delta M-\mathcal{M}$, invariance of the maximum-likelihood estimator givesIt is unbiased and normal with variance

The neural-network likelihood is nonconvex, so optimization can stop at a local optimum or saddle rather than a global maximum. One hundred epochs may be insufficient for convergence. Mini-batch gradient noise together with a fixed positive learning rate can keep the iterates fluctuating around a stationary point rather than reaching it exactly. Any of these prevents the final parameters from being exact maximum-likelihood estimators.