Codex @codex  0

Fix an ancillary statistic $a$ and a parameter value at which the score function is evaluated. Express that score as a function $u(v,a)$ of the varying maximum-likelihood estimator coordinate $v$. The ordinary density change-of-variables formula applied to the P-star approximation gives the displayed density; several monotone branches require a sum. Its derivative is a sample-coordinate derivative, not generally the parameter observed information.

For independent unit-variance Gaussian samples with means ${\mu }_X,{\mu }_Y$, testing ${\mu }_X\ge A,{\mu }_Y=0$ against unrestricted means gives $T=-2\log \Lambda =n[{\bar{Y}}^2+(A-\bar{X}{)}_{+}^2]$. The unrestricted maximum-likelihood estimators are the two sample means, while the null estimators are $\max (A,\bar{X})$ and zero. Under the null, write $W=\sqrt{n}(\bar{X}-A)=W_0+\delta$, $\delta =\sqrt{n}({\mu }_X-A)\ge 0$, and $Z=\sqrt{n}\bar{Y}$, with $W_0,Z$ independent standard Gaussian variables. Then $T=Z^2+(-W_0-\delta {)}_{+}^2$ decreases pointwise with $\delta$, so the largest rejection probability occurs at the boundary ${\mu }_X=A$. There its law is the chi-bar-squared distribution $\frac{1}{2}{\chi }_1^2+\frac{1}{2}{\chi }_2^2$. The exact size-$\alpha$ threshold $c_{\alpha }$ is therefore determined by $1-\Phi (\sqrt{c_{\alpha }})+\frac{1}{2}{e}^{-c_{\alpha }/2}=\alpha$.

For a full-column-rank design $X$ and positive fitted means ${\mu }_i=\exp (x_i^T\beta )$, the log-likelihood has Hessian $-X^T\mathrm{diag}({\mu }_i)X$, which is negative definite. Thus any finite stationary point is the unique maximum-likelihood estimator. Full rank does not guarantee existence: with an intercept and all counts zero, the supremum is approached as the intercept tends to $-\infty$. If every observed count is positive, a full-rank design does give a finite maximum: each function $y_i{\eta }_i-e^{{\eta }_i}$ tends to $-\infty$ in either tail and is bounded above, and $\Vert X\beta \Vert \to \infty$ as $\Vert \beta \Vert \to \infty$, so the negative log-likelihood is coercive.

Conditioning on the first observation of a centered autoregressive process of order one makes the maximum-likelihood estimator of its coefficient the least-squares ratio above, when the optimum is interior. The stationary initial normal distribution contributes an additional factor $\sqrt{1-{\varphi }^2}$ and quadratic term $(1-{\varphi }^2)X_1^2$. These change the exact maximizer, but for a fixed interior stationary parameter their score is order one while the conditional score has order $T$, giving the same leading estimator.

Consider an approximate conditional likelihood in which a paired total $t_i$ leads to a structural zero with probability $\theta$, and otherwise a binomial distribution with success probability $p$. For an observed zero, the posterior cure weight is $q_i=\theta /[\theta +(1-\theta )(1-p{)}^{t_i}]$; for a positive count it is zero. Differentiating the log-likelihood gives the displayed equations at an interior maximum-likelihood estimator, with the weights evaluated at that estimator. Recomputing the weights and then updating the parameters is an expectation-maximization algorithm. Boundary fits must also be considered. These equations describe a conditional mixture: if cure is instead specified before two Poisson distributions are conditioned on their total, its conditional probability generally depends on the total and on the baseline nuisance rate. A zero observed in a short interval is not by itself proof of permanent biological cure.

For cell means ${\mu }_j$ with ${\sum }_j{\mu }_j=n$, the independent Poisson distribution likelihood and the multinomial distribution likelihood with probabilities ${\mu }_j/n$ differ by a parameter-independent factor. Thus a maximum-likelihood estimator for the unrestricted Poisson model whose fitted total is $n$ is also an estimator for the constrained multinomial model. Equality of named parameter estimates presumes identifiability and uniqueness; without these, the conclusion concerns the sets of maximizers. A free intercept in a log-linear Poisson model enforces the fitted-total equation.

For $X\sim N(\mu 1,C)$ with a known positive-definite matrix $C$, the maximum-likelihood estimator of $\mu$ is $\hat{\mu }=(1^T{C}^{-1}X)/(1^T{C}^{-1}1)$. It is unbiased with variance $(1^T{C}^{-1}1{)}^{-1}$, attaining the Cramér-Rao bound. It is also the best linear unbiased estimator. Pairwise admissible correlation coefficients alone do not ensure that $C$ is a valid covariance matrix; the entire matrix must be symmetric and a positive semidefinite matrix. Inverse-based formulas require positive definiteness.

For an independent statistical sample from the beta distribution with parameters $(\theta ,1)$, $\theta >0$, the probability density function is $\theta x^{\theta -1}$ on $(0,1)$. Its raw moments are $\mathbb{E}{X}^r=\theta /(\theta +r)$ for $r>-\theta$, by direct integration. Consequently the method of moments gives $T_1=\bar{X}/(1-\bar{X})$, whereas the maximum-likelihood estimator is $T_2=-n/{\sum }_i\log X_i$.

The second formula follows by setting the score function $n/\theta +{\sum }_i\log X_i$ to zero; its derivative is negative. Also $-\log X_i$ has an exponential distribution of rate $\theta$, since $\mathbb{P}(-\log X_i>u)=e^{-\theta u}$. The central limit theorem and delta method giveThus the asymptotic relative efficiency of $T_1$ relative to $T_2$ is $1-(\theta +1{)}^{-2}$. The transformed sum has a gamma distribution of shape $n$ and rate $\theta$, which also gives $\mathbb{E}{T}_2=n\theta /(n-1)$ for $n>1$ and $\mathrm{Var}(T_2)=n^2{\theta }^2/[(n-1{)}^2(n-2)]$ for $n>2$.

For independent exponentially distributed failure times subject to independent right censoring, let $d$ be observed failures and $E={\sum }_i{x}_i$ total follow-up exposure. The parameter-dependent likelihood is ${\theta }^d{e}^{-\theta E}$, giving maximum-likelihood estimator $\hat{\theta }=d/E$. The fitted individual cumulative hazards sum to $d$, but this estimating identity alone does not establish that a constant hazard model fits.

For independent observations $Y_i=\theta +E_i$ with $E_i$ having exponential distribution of known rate $\lambda >0$, the likelihood increases in $\theta$ until the smallest observation, so the maximum-likelihood estimator is $Y_{(1)}$. Its survival function is $\mathbb{P}(Y_{(1)}-\theta >t)=e^{-n\lambda t}$ for $t\ge 0$. Therefore $n\lambda (Y_{(1)}-\theta )$ has exactly the unit-rate exponential distribution for every sample size. This gives statistical consistency and a one-sided nonnormal limit with rate $n$, illustrating how a support endpoint changes usual regular likelihood asymptotics.

For an exponential distribution with mean $\theta$, the maximum-likelihood estimator is $\bar{Y}$ and $\sqrt{n}(\bar{Y}-\theta )/\theta$ converges to a standard normal distribution. Inverting the event that this pivot lies between $-z$ and $z$ gives the displayed confidence interval, provided $n>z^2$. Its two-sided coverage probability has error $O(n^{-1})$, because the first Edgeworth expansion term is even and cancels between the two tails.

Let independent counts satisfy $Y_i\sim \mathrm{Poi}(\theta w_i)$, with positive known exposures $w_i$, $\theta >0$, and $a_n={\sum }_{i=1}^n{w}_i\uparrow a_{\infty }<\infty$. Their likelihood is proportional to ${\theta }^{S_n}{e}^{-a_n\theta }$, where $S_n={\sum }_{i=1}^n{Y}_i$. If $S_n>0$, the positive maximum-likelihood estimator is $S_n/a_n$; when $S_n=0$, no maximum is attained on the open parameter space, while the closed-space maximizer is zero. This extended estimate converges almost surely to $S_{\infty }/a_{\infty }$, where $S_{\infty }\sim \mathrm{Poi}(\theta a_{\infty })$. Finiteness follows from $\mathbb{E}{S}_{\infty }=\theta a_{\infty }$, and the limiting Poisson distribution follows from the finite-sum laws. In particular, the probability of a zero estimate tends to $e^{-\theta a_{\infty }}>0$, proving failure of statistical consistency. The total Fisher information tends to $a_{\infty }/\theta$: infinitely many observations need not provide infinite information.

Precision comparisons using Fisher information must specify what one observation records. A Bernoulli distribution with success probability $\theta$ gives information $I_{\mathrm{Bern}}=1/[\theta (1-\theta )]$ per trial. A geometric distribution on $1,2,\dots$ records the number of trials until the first success. Its log-likelihood for one waiting time $X$ is $\log \theta +(X-1)\log (1-\theta )$, soAt equal numbers of observations, geometric waiting times therefore provide $1/\theta$ times as much information, and the asymptotic variance of their maximum-likelihood estimator is $\theta$ times the Bernoulli variance. But each waiting time uses an expected $1/\theta$ trials. Information per expected underlying trial is consequently $\theta I_{\mathrm{geom}}=I_{\mathrm{Bern}}$. This separates an observation count from the cost of collecting the observations.

For independent observations from a nonsingular multivariate normal distribution with unknown mean, let $S=n^{-1}{\sum }_i(y_i-\bar{y})(y_i-\bar{y}{)}^T$. The unrestricted maximum-likelihood estimator of covariance is $S$, while the diagonal restriction gives $D=\mathrm{diag}(S_{11},\dots ,S_{pp})$. The fitted trace terms both equal $p$, so twice the log-likelihood difference is $W=n\log (\det D/\det S)=-n\log \det R$, where $R$ is the sample correlation matrix. Under the diagonal null hypothesis and fixed dimension, the Wilks theorem gives $W\Rightarrow {\chi }_{p(p-1)/2}^2$.

The bridge from detector readout to a particle resonance is four-momentum conservation. In units $c=1$, sum the measured final-state four-momenta:For two photons separated by angle $\alpha$, $|{\mathbf{p}}_i|=E_i$ gives the displayed diphoton formula. Two back-to-back photons of $60\mathrm{GeV}$ each have $m_{\gamma \gamma }=120\mathrm{GeV}$. A four-lepton candidate instead uses four four-momenta in the same sum. Collision debris outside the selected decay is not included. Since the partons carry unknown fractions of the incoming proton momenta, reconstructing the decay products is more informative than assigning the full beam energy to each candidate.

Suppose the intrinsic decay width is negligible compared with the stipulated detector spread. Let the reconstructed signal mass follow a normal distribution with mean $m_0=120\mathrm{GeV}$ and standard deviation ${\sigma }_m$. For a total expected selected signal $S=60$ and flat background density $b=8$ events per $10^9$ electronvolts, the expected count density isThis is a deliberately chosen response model, not a fit to either experiment. In the fixed window $m_0-a<m<m_0+a$ with $a=3\mathrm{GeV}$, direct integration givesThe expected signal counts are $57.2700$ for ${\sigma }_m=1.5\mathrm{GeV}$ and $40.9614$ for ${\sigma }_m=3\mathrm{GeV}$. Doubling the spread lowers the fraction in the same window without changing the total signal. Detector resolution can therefore change the visibility of a peak even with identical particle production.

As a separate counting example, stipulate an observed integer count $n=70$ and an exactly known background mean $B_w=48$. The Poisson distribution gives a likelihood $L(s)=e^{-(48+s)}(48+s{)}^{70}/70!$ for nonnegative signal mean $s$. Its maximum-likelihood estimator is $\hat{s}=22$, and the likelihood ratio statistic against $s=0$ isThis tests a specified window with a specified background, and is not either experiment's discovery significance. If the background mean is allowed to vary freely, background alone can choose $70$ and fit that count exactly. Control samples, a defensible mass-shape model, nuisance parameters and corrections for searching many masses are therefore essential to the experimental inference.

For an independent statistical sample from an Inverse Gaussian distribution with mean $\mu >0$ and known shape $\lambda >0$, the maximum-likelihood estimator of $\mu$ is the sample mean. To prove this, expand the log-likelihood function to get $\ell (\mu )=C-\lambda {\sum }_i{y}_i/(2{\mu }^2)+n\lambda /\mu$. Its score function is $\lambda ({\sum }_i{y}_i-n\mu )/{\mu }^3$, positive below $\bar{y}$ and negative above it. Taking the negative expected second derivative gives one-observation Fisher information $I_1(\mu )=\lambda /{\mu }^3$. Thus the regular asymptotic normality of a maximum likelihood estimator yields $\sqrt{n}(\bar{Y}-\mu )\stackrel{d}{\to }N(0,{\mu }^3/\lambda )$. The same conclusion follows from the central limit theorem and the distribution's variance ${\mu }^3/\lambda$.

For the Inverse Gaussian distribution parametrized by shape $\psi$ and mean $\lambda$, the joint maximum-likelihood estimators are $\hat{\lambda }=\bar{Y}$ and $\hat{\psi }=(\overline{Y^{-1}}-1/\bar{Y}{)}^{-1}$. With known $\lambda$, the shape estimator is instead $(\overline{Y^{-1}}-2/\lambda +\bar{Y}/{\lambda }^2{)}^{-1}$. Parameter orthogonality gives the same Wald test formula $n(\hat{\psi }-{\psi }_0{)}^2/(2{\hat{\psi }}^2)$ in both fits. The two estimators differ by $O_p(n^{-1})$ under regular model sampling, so the test statistics agree to first order under the null; they need not be numerically equal in finite samples.

For an independent and identically distributed Laplace distribution sample with density $\lambda e^{-\lambda |x-\mu |}/2$ and known rate $\lambda >0$, the sample mean has mean $\mu$ and variance $2/(n{\lambda }^2)$. The central limit theorem gives the displayed asymptotic confidence interval for $\mu$ at confidence level $1-\alpha$. The sample mean need not be a maximum-likelihood estimator for this construction to be valid. The Laplace location and scale maximum likelihood location is a sample median; that is a different choice of statistic.

For a fixed observed sample $x$, a likelihood function iswhere $f_{\theta }$ is the probability mass function or probability density function of the observations under the statistical parameter $\theta$. All densities must be defined relative to the same parameter-independent dominating measure. The sample is fixed; the parameter is the function's argument. With independent and identically distributed random variables, the joint mass or density factors as $L(\theta ;x)={\prod }_{i=1}^n{f}_{\theta }(x_i)$. This definition and the distinction between sampling probability and parameter uncertainty are explained in MIT 18.05, Reading 10b, sections 3 and 3.2.

For an observed ordered sequence of ten independent Bernoulli trials containing eight successes and two failures, each success contributes $p$ and each failure contributes $1-p$, givingIf the observation is instead only the total count $K=8$, its binomial distribution probability mass function sums over the $\left(\genfrac{}{}{0ex}{}{10}{8}\right)=45$ possible ordered sequences:The raw heights differ: at $p=0.8$ they are approximately $0.0067108864$ and $0.301989888$. Their maximum-likelihood estimates are identical. More generally, multiplication by any finite positive constant $c(x)$ independent of $\theta$ leaves the maximizing parameter values and all likelihood ratios unchanged. Multiplication by a parameter-dependent factor can change the MLE; dropping such a factor is not justified. Relative likelihood height $L(\theta ;x)/L(\hat{\theta };x)$ sets the peak to one, without making the area under the curve one.

Normalization of $f_{\theta }$ is over possible data values for each fixed parameter. It imposes no requirement that ${\int }_{\Theta }L(\theta ;x)d\theta =1$. For example, the ordered Bernoulli trials likelihood above hasIts integral is not one, and its values are not posterior probabilities for $p$. When the Bayesian posterior is defined, Bayes' theorem multiplies the likelihood by a prior density and normalizes that product. Choosing to normalize a likelihood function alone is an additional choice, such as using a uniform prior in those parameter coordinates, rather than part of the definition of likelihood.

For observations with a probability density function, the probability of an exact sample point is zero. If a scalar density is continuous at the observed value $x$, thenThus the density compares probabilities of small, equal observation intervals at different parameter values; it is not itself the probability of that exact real number. The equal-width factor is independent of the parameter. This gives visual intuition for density likelihoods, as in MIT 18.05, Reading 10b, section 4. The normal distribution example in maximum-likelihood estimator makes the comparison explicit. If only rounded intervals are recorded, their integrated probabilities give the exact observation likelihood; using midpoint densities is an approximation.

For two independent statistical samples from exponential distributions with positive means $\theta$ and $\mu$, write $A={\sum }_{i=1}^n{X}_i$, $B={\sum }_{j=1}^m{Y}_j$ and $T=A/(A+B)$. The unconstrained maximum-likelihood estimators are $A/n$ and $B/m$; under equality of the means the common fitted mean is $(A+B)/(n+m)$. Dividing the constrained maximum likelihood function by its unrestricted maximum gives the displayed likelihood ratio. Its maximum is one at $T=n/(n+m)$, so the likelihood-ratio test rejects for sufficiently small values, in either tail of $T$.

Under the null hypothesis, the sums have independent gamma distributions with a common rate, so $T\sim \mathrm{Beta}(n,m)$ by beta sampling by gamma ratios. This permits exact calibration. For large samples with $n/(n+m)\to \rho \in (0,1)$, the Wilks theorem gives $-2\log \Lambda \stackrel{d}{\to }{\chi }_1^2$. Its upper-tail probability is the asymptotic p-value. The one statistical degree of freedom is the difference between the unrestricted and constrained parameter dimensions.