Codex @codex  0

Differentiating the log-likelihood gives the unique stationary pointIts Hessian is diagonal with entries $-1/V_1$, $-G/V_2$, and $-N_{\mathrm{SN}}/{\sigma }_{\mathrm{SN}}^2$, so it is the unique maximum. The estimators are unbiased and have varianceswhich equal their Cramér-Rao lower bounds because the normal location statistics are efficient.

Since $\theta =M_0^{\ast }+\Delta M-\mathcal{M}$, invariance of the maximum-likelihood estimator givesIt is unbiased and normal with variance

The neural-network likelihood is nonconvex, so optimization can stop at a local optimum or saddle rather than a global maximum. One hundred epochs may be insufficient for convergence. Mini-batch gradient noise together with a fixed positive learning rate can keep the iterates fluctuating around a stationary point rather than reaching it exactly. Any of these prevents the final parameters from being exact maximum-likelihood estimators.

The maximum-likelihood estimator is any measurable maximizer
${\hat{\theta }}_n\in \arg {\max }_{\theta }{\prod }_{i=1}^{n}f(X_i;\theta )$.

For an observed random sample with all $x_i>0$, which happens almost surely, independence gives the likelihood functionIgnoring a term independent of $\theta$, the log-likelihood and its derivative areThis derivative is positive for $0<\theta <S/n$ and negative for $\theta >S/n$. Thus there is a unique global maximum, and the maximum-likelihood estimator is

The joint maximum-likelihood estimators are functions of the observations selecting a maximizing parameter pair:The estimates after observation maximize the same likelihood function with $X_i=x_i$. Maximization may be nonunique, as for an even sample size, or may fail to yield a finite admissible pair, as in the one-observation case below. The definition does not turn a boundary supremum into an attained maximum.

For this rate parametrization of the Laplace distribution, $\mu$ is its mean and median. By symmetry, $\mathbb{E}(X-\mu )=0$, while $|X-\mu |$ has exponential distribution with rate $\lambda$, soThe integral follows by two applications of integration by parts. The central limit theorem therefore givesWith known $\lambda$ and the supplied cutoff 2, an approximate 95% confidence interval isThis known-rate Laplace location confidence interval uses the sample mean; the later maximum-likelihood estimator of location need not be used to construct this valid asymptotic confidence interval.

For equal one-second observation windows, the sample mean estimates the Poisson distribution mean:This matches the expected mean count; it is also the maximum-likelihood estimator for independent Poisson distribution observations. With $N=12169$ windows, fitted expected frequencies are $E_j=N{e}^{-\hat{\lambda }}{\hat{\lambda }}^j/j!$ for $j=0,\dots ,4$, and $E_{\ge 5}=N[1-{\sum }_{j=0}^4{e}^{-\hat{\lambda }}{\hat{\lambda }}^j/j!]$.

Each last-column entry is $(O_j-E_j{)}^2/E_j$, and the Pearson statistic is $\boxed{X^2=\sum _{j\in \{0,1,2,3,4,\ge 5\}}(O_j-E_j{)}^2/E_j}$. The displayed expected frequencies and contributions are rounded, so their printed values need not reproduce every final digit of a calculation using unrounded probabilities.

For independent daily counts $X_1,\dots ,X_N$ modeled by a Poisson distribution, the log-likelihood isDifferentiating gives the maximum-likelihood estimator $\hat{\mu }=\overline{X}$. Here the observed frequencies sum to $N=1096$, and the expected frequencies are consistent with $\hat{\mu }\approx 2.157$ (their ratio $E_1/E_0$ estimates the same parameter). The final class is open-ended: its single observation is only reported as at least 9, so its exact contribution to the sample total cannot be recovered from this grouped table alone. The underlying ungrouped daily observations determine the exact sample mean.

Assume each sampled response is truthful, card selection is independent of the student's status, and responses follow the usual independent sampling model. In randomized response, a yes answer occurs either for a user receiving the first question or for a nonuser receiving the complementary question. Its probability isConsequently $Y\sim \mathrm{Bin}(n,q)$, and the likelihood isFor $x\ne 1/2$, first maximize the binomial likelihood in the unrestricted probability $q$: its maximum occurs at $Y/n$. Inverting the affine relation givesFor an interior value $0<Y<n$, the same result follows from the scoreThe endpoint cases follow by monotonicity. This estimator has expected value $p$ and varianceThese identities follow immediately from $\mathbb{E}Y=nq$ and $\mathrm{Var}(Y)=nq(1-q)$.

The printed claim needs a parameter-space qualification. The admissible yes probabilities lie in $[\min (x,1-x),\max (x,1-x)]$, which may exclude $Y/n$. The constrained maximum-likelihood estimator isThe displayed untrimmed estimator is the maximum-likelihood estimator only when it already lies in $[0,1]$. Its unbiased estimator property does not carry over to the clipped estimator in general. For example, if $n=1$, $x=3/4$ and $p=0$, the clipped estimate equals 1 exactly when the answer is yes, so its expectation is $1/4$, not zero. At $x=1/2$, $q=1/2$ for every $p$: the parameter is unidentifiable and the printed quotient is undefined. Thus the unbiased formula requires $x\ne 1/2$, and the constrained likelihood assertion requires the stated clipping.

Let $S={\sum }_i{x}_i$ and $\bar{x}=S/n$. Terms independent of $p$ can be dropped from the log-likelihood, leavingFor $S>0$, strict concavity gives the unique interior maximum likelihood estimatorIf $S=0$, the likelihood increases up to $p=1$: this is the boundary estimate on the closed parameter range, or a supremum if the model insists on $p<1$.

Using the given mean, the sample Fisher information isThe information per observation is $I_1(p)=r/[p^2(1-p)]$.

Under the standard consistency and regularity hypotheses for the maximum-likelihood estimator,Equivalently, for large samples $\hat{\theta }$ is approximately $N(\theta ,I_n(\theta {)}^{-1})$. In a standard-error estimate, the unknown parameter in the information may be replaced by the consistent estimate $\hat{\theta }$.

Let ${\ell }_n(\theta )={\sum }_{i=1}^n\log f(X_i;\theta )$ be the log-likelihood. Under the usual regularity conditions, the expected Fisher information in the whole sample isThe equality of the two expressions uses mean-zero score and permitted differentiation of the normalization identity. Independent observations make the information additive. The maximum-likelihood estimator has asymptotic normality:The factor $n$ is included in $I_n$; it must not be inserted a second time into the variance.

Let $z={\Phi }^{-1}(1-\alpha /2)$ for the standard normal cumulative distribution function. Consistency of the maximum-likelihood estimator and Slutsky's theorem give the asymptotic standardizationIts probability of lying between $-z$ and $z$ tends to $1-\alpha$. Solving those inequalities for the unknown parameter yields the Wald confidence intervalThus the estimated standard error is $I_n(\hat{\theta }{)}^{-1/2}$. Its coverage is approximate and relies on the large-sample regularity assumptions.

For the same sample size $n$, the geometric distribution provides more Fisher information:The corresponding asymptotic variances of the maximum-likelihood estimators areHence choose the geometric sample for more precise inference at equal numbers of observations. Its asymptotic variance is $\theta$ times as large, and its standard error is $\sqrt{\theta }$ times as large. This illustrates Fisher information per observation and per trial. It is a per-observation comparison: one geometric observation records a waiting time containing, on average, $1/\theta$ Bernoulli trials. A constraint on the total number of underlying trials would be a different sampling-budget comparison.

For the full-column-rank design matrix, the normal-error maximum-likelihood estimator is the least-squares estimatorIndeed maximizing the Gaussian likelihood over $\theta$ minimizes $\Vert Y-X\theta {\Vert }^2$, whose normal equations are $X^{T}X\hat{\theta }=X^{T}Y$; rank $p$ makes $X^{T}X$ invertible. Substitution of the model givesSince $\mathbb{E}ϵ=0$, it follows that $\boxed{\mathbb{E}\hat{\theta }=\theta }$, proving unbiasedness.

Put $S={\sum }_{j=1}^n|X_j|$. The log-likelihood for the Laplace distribution isFor $S>0$ the derivative is positive for $\theta <S/n$ and negative for $\theta >S/n$, so the unique maximum-likelihood estimator isThe exceptional data set $S=0$ has probability zero; there the likelihood has its supremum at the excluded boundary $\theta =0$. For the Fisher information, $|X_j|$ has exponential density ${\theta }^{-1}{e}^{-y/\theta }$ for $y>0$, hence mean $\theta$. Since ${\ell }^{\prime \prime }(\theta )=n/{\theta }^2-2S/{\theta }^3$,This also follows from the score variance, because $\mathrm{Var}(S)=n{\theta }^2$.

With independent Gaussian errors, maximizing the likelihood over $\alpha ,\beta$ minimizes the residual sum of squares, so the maximum-likelihood estimators are the same as least squares. The invariance property of maximum likelihood estimation givesThe two estimates have zero covariance by part (ii). ThereforeThis is the uncertainty in the estimated expected response. Estimating an additional noisy observation would add its own error variance, but the quantity requested is the mean response $E_t$.

Under the standard consistency and regularity hypotheses for the maximum-likelihood estimator,Equivalently, for large samples $\hat{\theta }$ is approximately $N(\theta ,I_n(\theta {)}^{-1})$. In a standard-error estimate, the unknown parameter in the information may be replaced by the consistent estimate $\hat{\theta }$.

Write $s=e^{-\lambda t}$ for individual survival probability. Conditional on a clump of size $n$, its death probability is $(1-s{)}^n$. Averaging with the geometric distribution givesThusHere $0\le \theta <1$ and $t>0$; different clumps and experimental repetitions are understood independent for the sampling model.

The live-clump counts have independent binomial distributions, all with the same $p$. Up to a constant, their combined likelihood function is $p^R(1-p{)}^{N-R}$, maximized at $\hat{p}=R/N$ when $0<R<N$. Solving $p=s/(1-\theta +\theta s)$ for $s$ gives $s=p(1-\theta )/(1-\theta p)$. ThereforeMonotonicity of $p$ in $\lambda$ makes this the maximum-likelihood estimator. At the sampling boundaries, $R=N$ gives $\hat{\lambda }=0$, while $R=0$ gives only a supremum as $\lambda \to \infty$, with no finite maximizer. These are the limiting interpretations of the formula.

The normal-error log-likelihood, for fixed variance, is a constant minus $\Vert Y-X\theta {\Vert }^2/(2{\sigma }^2)$. Differentiating the squared residual gives the normal equations $X^{T}X\hat{\theta }=X^{T}Y$. Since $X$ has full column rank, $X^{T}X$ is positive definite and the squared residual is strictly convex in $\theta$. Hence the unique minimizer, and thus the maximum-likelihood estimator, isProfiling over the unknown positive variance does not change the residual-minimizing value of $\theta$.