Codex @codex  0

For the normal linear model, the log-likelihood isThe full column rank of the design matrix makes $X^{T}X$ invertible. Differentiating in $\beta$ gives the normal equations $X^{T}X\hat{\beta }=X^{T}Y$. With the minimized residual sum of squares substituted, differentiation in ${\sigma }^2$ givesThese are the maximum-likelihood estimators almost surely; $RSS>0$ almost surely when ${\sigma }^2>0$ and $n>p$. The linear transformation of the multivariate normal distribution yieldsDefine the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. The fitted values are $\hat{Y}=HY=X\hat{\beta }$, the regression residuals are $e=Y-\hat{Y}=(I-H)Y$, and $RSS=e^{T}e$. The orthogonal projection $I-H$ has rank $n-p$ and annihilates $X\beta$. By Cochran's theorem,Consequently $E({\hat{\sigma }}^2)=(n-p){\sigma }^2/n$, with bias $-p{\sigma }^2/n$. The unbiased estimator isIts square root, the reported residual standard error, estimates $\sigma$; the assertion of unbiasedness applies to the variance, not generally to its square root.

For the paper-strength analysis, let $h_i$ be the measured percentage and $z_i=h_i-\overline{h}$. Both normal linear models use independent errors of common variance ${\sigma }^2$. Their mean functions are ${\beta }_0+{\beta }_1{z}_i$ for lm1, and ${\beta }_0+{\beta }_1{z}_i+{\beta }_2{z}_i^2$ for lm2, with separately fitted coefficients. The reported residual standard errors are $11.82$ and $4.42$, respectively.

For lm1, the estimated conditional expectation at a new percentage $x$ isThe original data mean $\overline{h}$ must be used for centering the new percentage; its numerical value is not supplied in the excerpt. Because ${\sum }_i{z}_i=0$, the two columns of this design matrix are orthogonal, and $\mathrm{Cov}({\hat{\beta }}_0,{\hat{\beta }}_1)=0$. Hence the coefficient standard errors in the output giveThis estimates uncertainty in the mean. Predicting an individual future batch would additionally require the new-error variance, estimated by $11.82^2$.

To compare the nested normal linear models, test $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$ within the quadratic model. The Student t-test statistic isIts two-sided p-value is $1.89\times 10^{-8}$. Equivalently $T^2$ is a partial F-test statistic with null distribution $F_{1,16}$. Reject the linear restriction and prefer lm2 at the 5% level. Its residual spread is much smaller; the increase in the coefficient of determination from $0.3054$ to $0.9085$ supports the same conclusion, although the test is the relevant complexity-adjusted comparison.

For regression diagnostics, inspect regression residuals against fitted means and hardwood percentage for omitted curvature, a scale-location plot for nonconstant variance, and a quantile-quantile plot against the normal distribution for departures from the error assumption. Check unusual observations using regression leverage and Cook's distance, and examine residuals in collection or batch order if dependence is plausible. Independence and a common variance require substantive justification as well as these plots; a small p-value for a polynomial term does not itself check the error model.

The Jeffreys prior is proportional to $[\theta (1-\theta ){]}^{-1/2}$, hence is $\mathrm{Beta}(1/2,1/2)$. Its posterior is $\mathrm{Beta}(X+1/2,n-X+1/2)$, givingThe interior case is used only when $n\ge 2$. Generally neither rule equals $X/n$: the mean agrees only at $X=n/2$; the mode agrees at the endpoints and, in the interior, at $X=n/2$. For the small experiments $n=1,2$, all possible observations happen to be among these cases, so the whole mode rule then coincides with the maximum-likelihood estimator.

The uniform prior has $s=1$, so the posterior is $\mathrm{Beta}(X+1,n-X+1)$. Its mean and mode areThus the uniform-prior posterior mode is the maximum-likelihood estimator, including $X=0,n$. The mean shrinks toward $1/2$ and equals the maximum-likelihood estimator only when $X=n/2$.

Writing $y_i$ for the observed counts, independence of the Poisson distributions gives the log-likelihoodIts score function and negative second derivative areUnless all $x_i$ vanish, $J(\beta )>0$, so the log-likelihood is strictly concave. Use Newton method, starting from a finite ${\beta }_0$, withStop when the score function or the change in $\beta$ is small; a reduced step can ensure that the log-likelihood increases. Any finite zero of the score function is the unique maximum-likelihood estimator.

Existence needs a qualification. If the nonzero $x_i$ include both signs, $U$ decreases from $+\infty$ to $-\infty$, so a finite root exists. If all nonzero $x_i$ are positive, a finite root exists exactly when ${\sum }_i{x}_i{y}_i>0$; otherwise the supremum is approached as $\beta \to -\infty$. The corresponding condition for all negative $x_i$ is ${\sum }_i{x}_i{y}_i<0$, with a supremum at $+\infty$ otherwise. When all $x_i=0$, the parameter is unidentifiable. These cases explain when the iterative algorithm actually has a finite estimate to find.

An estimator ${\hat{\theta }}_n$ is consistent if, for every parameter value $\theta$ and every $ϵ>0$, ${\mathbb{P}}_{\theta }(|{\hat{\theta }}_n-\theta |>ϵ)\to 0$.

For the proposed root criterion, fix $ϵ>0$. Convergence in probability at the two points givesOn this event the intermediate value theorem gives a zero between the two points. Since the zero is unique, it is ${\hat{\theta }}_n$. Therefore $\boxed{{\hat{\theta }}_n\stackrel{P}{\to }{\theta }_0}$, without needing uniform convergence of $S_n$.

For independent $N({\theta }_0,1)$ observations, the log-likelihood has derivative $n({\bar{X}}_n-\theta )$ and negative second derivative $-n$. Its unique maximizer is ${\hat{\theta }}_n={\bar{X}}_n$. Apply the criterion to $S_n(\theta )=\theta -{\bar{X}}_n$: the strong law of large numbers gives the limit $S(\theta )=\theta -{\theta }_0$, with the required signs. Hence the maximum-likelihood estimator is consistent.

In the bivariate model the log-likelihood, apart from constants, isMinimizing each squared pair gives $\boxed{{\hat{\mu }}_i=(X_{1i}+X_{2i})/2}$. Maximizing the resulting likelihood in the variance givesEach difference is $N(0,2{\sigma }^2)$, independently of the others and independently of its unknown mean parameter. Thus the strong law of large numbers gives ${\hat{\sigma }}^2\to {\sigma }^2/2$ in probability, not ${\sigma }^2$. This is the Neyman-Scott incidental parameter problem: the number of nuisance means grows with the sample size, but each mean is estimated from only two observations. The usual fixed-dimensional likelihood consistency theory therefore does not apply. The corrected estimator $2{\hat{\sigma }}^2$ is consistent.

This is a generalized linear model with independent Bernoulli distributions for the responses and the logit link. Under treatment coding with colour one as the reference level, the linear predictor isColour is a categorical variable, not a numerical trend. Width has the same slope for every colour, since no interaction term is fitted. The parameters are estimated by maximum likelihood estimation.

The approximate Wald confidence interval for the width coefficient is $\boxed{0.46\pm 1.96(0.106)}$, or approximately $(0.2522,0.6678)$. This uses the approximate normal distribution of the maximum-likelihood estimator, rather than a confidence interval for an individual binary response.

For the requested crab, the fitted linear predictor is $\hat{\eta }=-11.38-0.22+0.46(20)=-2.40$. Applying the inverse logit link gives $\boxed{\hat{p}=(1+e^{2.40}{)}^{-1}\simeq 0.0832}$. The rounded displayed coefficients need not reproduce the reported test statistics exactly.

Let $Q={\sum }_i{x}_i^2>0$. Under the usual nondegenerate regression assumption $n>2$, maximizing the normal likelihood is equivalent to minimizing the residual sum of squares. The normal equations and ${\sum }_i{x}_i=0$ give the maximum-likelihood estimatorsWrite $A=\overline{\epsilon }$ and $B={\sum }_i{x}_i{\epsilon }_i/Q$. Then $\hat{\alpha }=\alpha +A$, $\hat{\beta }=\beta +B$, and the residual is $r_i={\epsilon }_i-A-x_{i}B$. Independence of the errors givesFor each $i$,Thus the three requested variables are pairwise uncorrelated.

They are linear functions of the jointly normal error vector. More strongly, the whole vector $(A,B,r_1,\dots ,r_n)$ has a multivariate normal distribution, and its block covariances show that $A$, $B$ and the residual vector are independent, by uncorrelated jointly normal variables are independent. Independence is preserved when applying a measurable function to the residual block. HenceThe residual projection is $I-11^{\mathsf{T}}/n-x{x}^{\mathsf{T}}/Q$, of rank $n-2$, so additionally ${\sum }_i{r}_i^2/{\sigma }^2\sim {\chi }_{n-2}^2$. This is the orthogonal projection of a Gaussian vector interpretation of the same result. The printed design conditions alone allow $n=2$; then the fit is exact, the positive-variance likelihood has no finite maximizer, and the residual test in part (b) is undefined. The standard testing question therefore needs $n>2$.

Put ${\mathrm{RSS}}_1={\sum }_i(Y_i-\hat{\alpha }-\hat{\beta }{x}_i{)}^2$ for the unrestricted fit. Under $H_0$, the best slope is still $\hat{\beta }$ because the design is centered. Orthogonality of the residuals to the constant vector givesAt either fitted mean, the maximum-likelihood estimator of variance is $\mathrm{RSS}/n$. The ratio used by the generalized likelihood-ratio test is consequentlySmall likelihood ratios correspond to large $|T_n|$. Under $H_0$, part (a) gives independent $\sqrt{n}\overline{Y}/\sigma \sim N(0,1)$ and ${\mathrm{RSS}}_1/{\sigma }^2\sim {\chi }_{n-2}^2$. Thus $T_n$ has the Student's t-distribution with $n-2$ degrees of freedom. The exact size-5% critical region iswhere $t_{\nu ,0.975}$ is the 97.5th percentile of that distribution. The denominator uses the unbiased residual variance estimate, not the likelihood variance denominator $n$.

Under an alternative, the same statistic has the representationThe joint law of $Z,W$ is parameter-free, and $\beta$ has disappeared. Thereforewhere $g(d)=\mathbb{P}(|(Z+\sqrt{n}d)/\sqrt{W/(n-2)}|>t_{n-2,0.975})$. This is also the noncentral t-distribution description of the power. As explained in part (a), an exact residual-based test requires $n>2$.

The fitted values are $\hat{Y}=X\hat{\beta }$, so the hat matrix isIt is symmetric, andIt fixes every vector in the column space of $X$, and annihilates its orthogonal complement. Thus $H$ is the orthogonal projection onto that column space, has matrix rank and trace $p$, and $I-H$ is the complementary orthogonal projection. ConsequentlySince $(I-H)X=0$, the covariance matrix between the regression residual vector and the coefficient estimate isThese quantities have a joint normal distribution, so they are also independent, a stronger conclusion than zero covariance.

Now $\hat{e}=(I-H)\epsilon$ has mean zero. Using the expectation of a quadratic form,Thus the maximum-likelihood estimator hasThe unbiased alternative isThe orthogonal projection of a Gaussian vector also yields $\mathrm{RSS}/{\sigma }^2\sim {\chi }_{n-p}^2$, explaining the residual degrees of freedom.

Assume the known covariance matrix $\Sigma$ is positive definite. The Gaussian likelihood, up to constants, isIts gradient is $X^T{\Sigma }^{-1}(Y-X\beta )$. The negative Hessian matrix $X^T{\Sigma }^{-1}X$ is positive definite because $X$ has full column rank. Thus the unique maximum-likelihood estimator is the generalized least squares estimatorIt is unbiased, and multiplication of its linear coefficient matrix by $\Sigma$ givesIn fact ${\hat{\beta }}_{\mathrm{GLS}}\sim N_p(\beta ,(X^T{\Sigma }^{-1}X{)}^{-1})$. Equivalently, a whitening transformation premultiplies $Y$ and $X$ by ${\Sigma }^{-1/2}$, after which ordinary least squares applies. There is no extra ${\sigma }^2$ factor when $\Sigma$ is the complete known covariance; such a factor appears if only a shape matrix $\Omega$ is known and $\Sigma ={\sigma }^2\Omega$.

For observation $X=k$, the likelihood is proportional to $p^k(1-p{)}^{n-k}$. For $0<k<n$, its logarithmic derivative is $k/p-(n-k)/(1-p)$, vanishing uniquely at $k/n$; the logarithm is strictly concave. At $k=0$ or $k=n$, monotonicity gives the boundary maximum. Thus the maximum-likelihood estimator isA uniform prior multiplies this likelihood by a constant. The beta-integral normalization therefore gives the posterior distributionIn particular the posterior mean $(k+1)/(n+2)$ is not automatically the optimal estimator for the weighted loss in the next part.

The support restricts $\theta$ to $\theta >0$, $\theta \le X_{(1)}$ and $\theta \ge X_{(n)}-1$. Within its feasible interior, the likelihood function is proportional to ${\prod }_i(X_i-\theta {)}^{-\alpha }$ and is strictly increasing in $\theta$, diverging as $\theta \uparrow X_{(1)}$. Under the printed infinite-endpoint-density, or extended-likelihood convention, the maximum-likelihood estimator isA density is defined only almost everywhere. If its value at the singular endpoint is reassigned a finite value, a finite maximum need not exist; the sample-minimum answer uses the stated endpoint/supremum convention.

Independence and $\mathbb{P}(Y\le t)=t^r$ on $[0,1]$ giveThe error is positive almost surely, so the estimator is not an unbiased estimator. More explicitly its bias is

With $\Sigma =I_d$, constrained likelihood maximization minimizes $\Vert \theta -\overline{X}{\Vert }^2$. On the unit sphere this is $1+\Vert \overline{X}{\Vert }^2-2{\theta }^T\overline{X}$. By the Cauchy-Schwarz inequality its minimizer isIf $\overline{X}=0$, every unit vector is a maximum-likelihood estimator; a specified measurable tie-breaking choice makes an estimator defined everywhere. This event has probability zero in the nonsingular normal model, but the optimization still has that exceptional case.

The maximum-likelihood estimator isUnder the score-outer-product convention stated in the question,This is the sample covariance matrix with divisor $n$. The strong law of large numbers, applied entrywise, givesThus the empirical score outer-product information consistently estimates the per-observation Fisher information.

Put $w_k=C_k^{-1}$, $u_i=H_i^{-1}$, $A={\sum }_k{w}_k$, $B={\sum }_i{u}_i$, and form the weighted meansThe two equations obtained from the score function areConsequently the maximum-likelihood estimators areThe Hessian matrix of the log likelihood isIts first leading principal minor is negative and its determinant is $AB>0$, so it is a negative-definite matrix. Thus the stationary point is the unique global maximum.

For $X_1\sim \mathrm{Poisson}(\theta )$, its probability generating function isThus $\mathbb{E}[X_1]=G^{\prime }(1)=\theta$ andThe log-likelihood is, up to an additive constant,so the maximum-likelihood estimator is ${\hat{\theta }}_{\mathrm{MLE}}=\overline{X}$. The central limit theorem gives

Under the null model, $\pi =1$, which is a boundary point of the parameter space. Standard interior-point asymptotic normality of the maximum-likelihood estimator therefore need not apply.

Indeed, put $a=1-e^{-\lambda }$. Under $\pi =1$, $N_{+}\sim \mathrm{Binomial}(n,a)$. For the unconstrained estimator $\tilde{\pi }=N_{+}/(na)$, the central limit theorem givesSince $\hat{\pi }=\min \{1,\tilde{\pi }\}$, the continuous mapping theorem yieldsThis distribution has probability $1/2$ at zero and a continuous normal half-density on the negative half-line, so it is not normal.

Let $S_n={\sum }_{i=1}^n{X}_i$. The log-likelihood iswith score function $n/\theta -S_n$ and second derivative $-n/{\theta }^2<0$. The unique maximum-likelihood estimator is thereforeFor an exponential distribution of rate $\theta$,The central limit theorem givesApplying the delta method to $h(x)=1/x$, for which $h^{\prime }(1/\theta )=-{\theta }^2$, yields the exponential-rate maximum-likelihood estimator limit

For observations $x=(x_1,\dots ,x_n)$, the likelihood function isA maximum-likelihood estimator is any measurable choiceFor one observation in a regular scalar model, the score function isand the Fisher information iswhere the second identity requires the usual differentiation-under-the-integral regularity. For $n$ independent observations, Tensorization of Fisher information gives $I_n(\theta )=nI(\theta )$.