Codex @codex  0

For an independent normal distribution sample with unknown mean, test ${\sigma }^2={\sigma }_0^2$ against unrestricted positive variance. Put $q={\sum }_i(x_i-\bar{x}{)}^2/{\sigma }_0^2$ and assume $n\ge 2$. Profiling the mean and variance gives the likelihood ratio $\lambda (q)=(q/n{)}^{n/2}{e}^{(n-q)/2}$ for a nondegenerate sample. Its logarithmic derivative $\frac{1}{2}(n/q-1)$ changes sign at $n$, so small ratios select a lower and an upper tail.

Under the null hypothesis, Cochran's theorem gives $Q\sim {\chi }_{n-1}^2$. The exact size-$\alpha$ cutoffs $0<q_1<n<q_2$ therefore satisfyThe second equation makes the two boundary ratios equal. Rejection on $q\le q_1$ or $q\ge q_2$ is consequently the exact likelihood-ratio test, and its tails need not have equal probability. For a large sample, the Wilks theorem instead calibrates $W=-2\log \lambda$ by ${\chi }_1^2$. A Bartlett correction for a normal variance with unknown mean improves this asymptotic calibration; it is unnecessary when using the exact $Q$ distribution.

For independent samples from normal distributions with separate unknown means, test equal variances against ${\sigma }_X^2>{\sigma }_Y^2$. Profiling the means gives sample means. Under the null the variance maximum is $(S_X+S_Y)/(n_X+n_Y)$; the separate maxima are $S_X/n_X$ and $S_Y/n_Y$. If the latter obey the proposed order, their likelihood quotient isIt decreases strictly as $(S_X/n_X)/(S_Y/n_Y)>1$ increases. Otherwise the ordered alternative maximum lies on the equal-variance boundary and the quotient is one. Under the null, Cochran's theorem gives independent chi-squared residual sums with degrees $n_X-1,n_Y-1$, so rejection is in the upper tail of the displayed F-distribution statistic, calibrated at the desired significance level.

Write $1$ for the all-ones vector and setThe centering assumption makes $P_0{P}_X=0$. Thus the full normal linear model has fitted-value orthogonal projection $P_0+P_X$, and its residual sum of squares is $R=Y^{\mathsf{T}}(I-P_0-P_X)Y$. Assume $\nu >0$, as is needed to estimate the unknown error variance from residuals.

For the null that all slopes vanish, the reduced fitted value is $\bar{Y}1$, with residual sum of squares $R_0=Y^{\mathsf{T}}(I-P_0)Y$. The F-test isReject for an upper-tail value exceeding the $1-\alpha$ quantile of this F-distribution. The required theorem is Cochran's theorem: for an isotropic normal distribution, projections onto orthogonal subspaces are independent, and the squared norm on a rank-$r$ subspace divided by ${\sigma }^2$ has a chi-squared distribution with $r$ degrees of freedom. Here the null makes the projected mean in $\mathrm{col}(X)$ zero, so $(R_0-R)/{\sigma }^2\sim {\chi }_p^2$, independently of $R/{\sigma }^2\sim {\chi }_{\nu }^2$.

Let $p_1,p_2$ be the numbers of columns in the two predictor blocks. To test the first block while retaining the second as nuisance, formThe partial F-test for nested linear models isAgain reject in the upper tail. The difference of the nested fitted-value projections has rank $p_1$, is orthogonal to the full residual projection, and annihilates the null mean, which gives the same independent chi-squared argument. If a block has no columns, the corresponding test is vacuous.

The phrase orthogonal statistical parameters refers to zero cross-block Fisher information, not to the numerical parameter vectors being perpendicular. In this model the cross-block information is $X_1^{\mathsf{T}}{X}_2/{\sigma }^2$, so the orthogonal coefficient blocks in a centered normal linear model condition isThen the two fitted subspaces are orthogonal, the block least-squares estimators have zero covariance and are independent because they are jointly Gaussian, and each block's extra sum of squares is unchanged by including the other block. Centering also makes both blocks orthogonal to the intercept.

Assume the two normal samples are independent. Write $n_X=6$, $n_Y=21$, and let $S_X,S_Y$ denote their residual sums of squares about the respective sample means. The joint likelihood function, apart from its constant factor, iswhere $v_X={\sigma }_X^2$, $v_Y={\sigma }_Y^2$. For every fixed variance pair the means maximize this likelihood at $\bar{X},\bar{Y}$, because $\sum (X_i-{\mu }_X{)}^2=S_X+n_X(\bar{X}-{\mu }_X{)}^2$, and likewise for $Y$.

Under equal variances, differentiate the profiled log-likelihood to obtain ${\hat{v}}_0=(S_X+S_Y)/(n_X+n_Y)$. Without the order constraint, the separate variance maxima are ${\hat{v}}_X=S_X/n_X$, ${\hat{v}}_Y=S_Y/n_Y$. If ${\hat{v}}_X>{\hat{v}}_Y$ they lie in the alternative. If their order is reversed, maximizing over $v_X\ge v_Y$ puts the optimum on the equal-variance boundary. These assertions also follow by strict concavity in the precisions $1/v_X,1/v_Y$.

Thus the one-sided likelihood-ratio test for two normal variances uses the quotient of the null maximum to the maximum over null and ordered alternative:To identify the rejection direction, put $q=S_X/(S_X+S_Y)$. Up to constants, $\log \Lambda =(n_X/2)\log q+(n_Y/2)\log (1-q)$. Its derivative is $[n_X-(n_X+n_Y)q]/[2q(1-q)]$, negative on the ordered branch $q>n_X/(n_X+n_Y)$. Hence small likelihood ratios mean a large sample variance ratio.

The null law must account for estimating both means. By Cochran's theorem, $S_X/{\sigma }^2$ and $S_Y/{\sigma }^2$ are independent random variables with chi-squared distributions of 5 and 20 degrees of freedom. Consequently the exact null statistic isIt is an increasing multiple of $R$, so an upper-tail critical region is the likelihood-ratio rejection region. The observed statistic is $F=(30/5)/(40/20)=3$. The supplied 95th percentile is 2.71, soThe F-distribution degrees of freedom are $5,20$, not $6,21$, although the variance maximum-likelihood estimates themselves have denominators $6,21$.

Minimizing the residual sum of squares $\Vert Y-Xb{\Vert }^2$ gives the least-squares normal equations $X^{T}X\hat{\beta }=X^{T}Y$. Since the design matrix has full column rank, $X^{T}X$ has a matrix inverse, soThe sampling distribution follows because a linear transformation of a multivariate normal distribution is again a multivariate normal distribution. Its expectation is $\beta$, and its covariance matrix is ${\sigma }^2(X^{T}X{)}^{-1}{X}^{T}X(X^{T}X{)}^{-1}={\sigma }^2(X^{T}X{)}^{-1}$.

The hat matrix is $H=X(X^{T}X{)}^{-1}{X}^T$. It is the orthogonal projection matrix onto the column space of $X$: $H^T=H$, $H^2=H$ and $HX=X$. Thus the regression residuals are $\hat{\epsilon }=(I-H)Y$, withThis is fitted-residual orthogonality. The vectors are jointly normal, so independence of uncorrelated jointly normal variables strengthens both zero-covariance statements to independence.

For the quadratic linear regression, the columns of the design matrix are $1,x,x^2$. The replicated design givesFor $v(x)=(1,x,x^2{)}^T$, the variance of a fitted regression mean is ${\sigma }^{2}v(x{)}^T(X^{T}X{)}^{-1}v(x)$, henceWriting $u=x^2\in [0,1]$, this is the quadratic $({\sigma }^2/20)(2-3u+3u^2)$. Its derivative vanishes at $u=1/2$; it decreases before that point and increases afterwards. Therefore the maximum is ${\sigma }^2/10$ at $x=-1,0,1$, and the minimum is ${\sigma }^2/16$ at $x=\pm 1/\sqrt{2}$.

At $100^{\circ }\mathrm{C}$, $x=0$ and the expected yield is $\alpha$. The unbiased residual variance estimate is $s^2=2.43/(30-3)=0.09$. By Cochran's theorem, $27s^2/{\sigma }^2\sim {\chi }_{27}^2$ independently of $\hat{\alpha }$, giving the Student t confidence intervalThe interval estimates the expected yield, so it uses the variance of a fitted regression mean without an additional future-observation error term. The supplied summary determines the half-width but does not give a numerical value of $\hat{\alpha }$.

The Gaussian log likelihood function, apart from a constant, isDifferentiating in $\beta$ gives $X^{\mathsf{T}}(Y-X\hat{\beta })=0$. Full column rank makes $X^{\mathsf{T}}X$ invertible, soThe distribution follows by applying this linear map to the Gaussian error vector. Differentiating the profiled likelihood function in ${\sigma }^2$ givesFor its distribution, $P=X(X^{\mathsf{T}}X{)}^{-1}{X}^{\mathsf{T}}$ is symmetric idempotent, and the residual is $(I-P)ϵ$. The needed form of Cochran's theorem says that quadratic forms in a standard Gaussian vector associated with orthogonal symmetric projection matrices summing to identity are independent random variables with chi-squared distributions with degrees equal to their ranks. Applying it to $P$ and $I-P$ givesThus the maximum-likelihood variance estimate uses divisor $n$ and has expectation $(n-p){\sigma }^2/n$; the unbiased residual estimate would use divisor $n-p$. The fitted coefficient and residual quadratic form are also independent.

A useful full statement is the rank-sum form of Cochran's theorem. Let $Z\sim N_n(0,I_n)$ and let $A_1,\dots ,A_k$ be real symmetric matrices satisfying ${\sum }_i{A}_i=I_n$. Put $r_i=\mathrm{rank}(A_i)$ and $Q_i=Z^T{A}_{i}Z$. Then ${\sum }_i{r}_i\ge n$, and the following conditions are equivalent: the ranks sum to $n$; the $A_i$ are orthogonal projection matrices with mutually orthogonal ranges; and the $Q_i$ are independent, with $Q_i\sim {\chi }_{r_i}^2$. A rank-zero form is identically zero. In particular, the usual projection version of Cochran's theorem partitions the squared length of a standard normal vector into independent chi-squared distributions.

Here is a proof which also explains the rank condition. Let $L_i=\mathrm{im}{A}_i$. Since every vector satisfies $x={\sum }_i{A}_{i}x$, the spaces $L_i$ span ${\mathbb{R}}^n$. Therefore their dimensions sum to at least $n$. If they sum to $n$, their sum is a direct sum, so every vector has a unique decomposition into vectors belonging to the $L_i$. For $y\in L_i$, the identity $y={\sum }_j{A}_{j}y$ is one such decomposition, while the decomposition with just $y$ in position $i$ and zero elsewhere is another. Uniqueness gives $A_{j}y={\delta }_{ij}y$. ThusA symmetric matrix with this idempotence is an orthogonal projection matrix. Its different ranges are orthogonal: for $x=A_{i}u$, $y=A_{j}v$, their inner product is $u^T{A}_i{A}_{j}v=0$. Choose an orthonormal basis for each range and concatenate them. The resulting change-of-basis matrix is an orthogonal matrix. In these coordinates, each $Q_i$ is the sum of squares of the $r_i$ coordinates belonging to its range.

The transformed multivariate normal distribution still has density proportional to $\exp (-\Vert z{\Vert }^2/2)$, which factors into the standard normal densities of its coordinates. The coordinates are independent, so the disjoint groups of squared coordinates give independent chi-squared distributions of the asserted degrees of freedom. This proves the forward implication. Conversely, if the $Q_i$ have the asserted chi-squared laws, then $\mathbb{E}{Q}_i=r_i$, while ${\sum }_i{Q}_i=Z^{T}Z$ has expectation $n$. Taking expectations yields ${\sum }_i{r}_i=n$. This completes the equivalence and the proof of Cochran's theorem.

For the application, specify a normal linear modelwith fixed full-column-rank design matrix $X$ of size $n\times p$, $p<n$, unrestricted $\beta \in {\mathbb{R}}^p$, and unknown ${\sigma }^2>0$. Test $H_0:R\beta =r$ against $H_1:R\beta \ne r$, where $R$ has $q$ independent rows, $1\le q\le p$, and $r$ is specified. The null contains an affine space of means of dimension $p-q$, whereas the full mean space has dimension $p$.

Let $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$ be the ordinary least squares and maximum-likelihood estimate in the full model, and write $G=(X^{T}X{)}^{-1}$. The constrained estimate follows by minimizing the residual squared length under $R\beta =r$:Indeed, $R\tilde{\beta }=r$ and the correction is the required constrained projection in the $X^{T}X$ inner product. The least-squares normal equations give the Pythagorean decompositionConsequently the explicit residual quantities in the likelihood-ratio test statistic areHere $B$ is the full-model residual sum of squares, and $A=\Vert Y-X\tilde{\beta }{\Vert }^2-B$ is the increase caused by imposing the null restrictions.

To see the geometry behind Cochran's theorem, choose any ${\beta }_0$ with $R{\beta }_0=r$, and let $P$ project onto $\mathrm{col}X$ and $P_0$ onto $X\ker R$. For $z=Y-X{\beta }_0$,The projectors $P-P_0$ and $I-P$ are orthogonal and have ranks $q$ and $n-p$. Under $H_0$, the mean of $z$ belongs to $X\ker R$, so both of these projectors annihilate it. Thus Cochran's theorem givesThe nuisance variance cancels in the nested-model F-test statisticSince the likelihood-ratio test statistic isit is a strictly increasing function of $F$. Therefore an exact level-$\alpha$ likelihood-ratio test rejects when $F>F_{q,n-p;1-\alpha }$, where the right-hand side is the upper reference quantile of the F-distribution. This calibration is exact under the normal linear model, rather than an asymptotic chi-squared replacement.

Assume $n\ge 2$, so the residual sum of squares is positive almost surely. Write $\overline{Y}=n^{-1}{\sum }_i{Y}_i$ and $S={\sum }_i(Y_i-\overline{Y}{)}^2$. The log-likelihood isThe decomposition ${\sum }_i(Y_i-\mu {)}^2=S+n(\overline{Y}-\mu {)}^2$ shows that, at every fixed positive variance, its constrained maximum-likelihood estimate of the mean is $\overline{Y}$. Substituting this mean and differentiating with respect to the variance givesThe denominator $n$ is required by maximum likelihood estimation; $S/(n-1)$ is the unbiased variance estimate, which is a different estimator.

Under the null variance ${\sigma }_0^2$, maximize over the same mean $\overline{Y}$. Subtracting the null maximum log-likelihood from the unrestricted maximum gives the likelihood-ratio test statisticBy Cochran's theorem, the residual sum of squares in a normal sample satisfies $U=S/{\sigma }_0^2\sim {\chi }_{n-1}^2$ under the null. Set $V=(U-n)/n$, so $U/n=1+V>0$. ThenThe normal mean is a nuisance parameter, but this null law does not depend on it.

A Bartlett correction removes the first-order discrepancy between the null mean of a regular likelihood-ratio test statistic and the degrees of freedom of its limiting chi-squared distribution. In the convention used here, if the limiting degrees of freedom are $d$ andthen $b$ is the Bartlett correction coefficient, also called the Bartlett correction factor, and the Bartlett-corrected likelihood-ratio statistic is $W_B=W/(1+b/n)$. Its null mean is $d+O(n^{-2})$. The finite-$n$ divisor is $1+b/n$, rather than $b$ itself. There is one restriction on the variance here, so $d=1$.

Expand the logarithm around $V=0$ and use the permitted termwise asymptotic integration:Using the supplied raw moments of the chi-squared distribution and expanding $(U-n{)}^r$ givesFor instance, $\mathbb{E}U=n-1$ and $\mathbb{E}{U}^2=(n-1)(n+1)$ give the first formula after subtracting $2n\mathbb{E}U-n^2$. The same binomial expansion, using the raw third and fourth moments, gives the other two formulas.

We must justify why higher moments cannot change the $n^{-1}$ coefficient. Represent $U={\sum }_{i=1}^{n-1}{Z}_i^2$ for independent standard normal variables, and put $T_i=Z_i^2-1$. Each $T_i$ is centered and has all absolute moments finite. With $H={\sum }_{i=1}^{n-1}{T}_i$, one has $V=(H-1)/n$. The supplied centered-sum bounds say, for each fixed integer $j\ge 2$,Also $\mathbb{E}H=0$ and $\mathbb{E}{H}^0=1$. Therefore, for every fixed $r\ge 2$, expanding $(H-1{)}^r$ givesThese are bounds on the absolute values of signed moments; the odd-moment estimate must not be misread as a bound on $\mathbb{E}|H{|}^j$. In particular, each fixed Taylor term of degree $r\ge 5$ has $n\mathbb{E}{V}^r=O(n^{-2})$ or smaller. Combined with the question's permission to integrate the asymptotic expansion term by term, this controls the higher-order contribution at the required order. It does not require a globally convergent Taylor series for every realized $V$.

Substituting the first three moment formulas now givesThus the Bartlett correction for a normal variance with unknown mean isIts null mean is $1+O(n^{-2})$, and the usual reference law is ${\chi }_1^2$. Keeping the estimated mean's loss of one residual degree of freedom, and using the maximum-likelihood variance divisor $n$, are both essential to the coefficient $11/6$.

Assume $n>p\ge 1$, the ordinary nonsaturated Gaussian linear model. The log likelihood is, up to a constant,The normal equations and full column rank give the least-squares estimator $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$. Write $H=X(X^{T}X{)}^{-1}{X}^T$ and $Q=I-H$, an orthogonal projection of rank $n-p$. Maximizing in ${\sigma }^2$ yieldsSince $QX=0$, the residual sum of squares is ${\epsilon }^{T}Q\epsilon$. Expanding its expectation uses only $\mathbb{E}{\epsilon }_i{\epsilon }_j={\sigma }^2{\delta }_{ij}$:This does not use Cochran's theorem. It proves bias $-p{\sigma }^2/n$ and givesIn the saturated case $n=p$, residuals vanish identically, the likelihood is unbounded as the positive variance tends to zero, and the unbiased residual estimator is unavailable. Thus the usual variance-estimation request requires $n>p$.

For the normal linear model, the log-likelihood isThe full column rank of the design matrix makes $X^{T}X$ invertible. Differentiating in $\beta$ gives the normal equations $X^{T}X\hat{\beta }=X^{T}Y$. With the minimized residual sum of squares substituted, differentiation in ${\sigma }^2$ givesThese are the maximum-likelihood estimators almost surely; $RSS>0$ almost surely when ${\sigma }^2>0$ and $n>p$. The linear transformation of the multivariate normal distribution yieldsDefine the hat matrix $H=X(X^{T}X{)}^{-1}{X}^T$. The fitted values are $\hat{Y}=HY=X\hat{\beta }$, the regression residuals are $e=Y-\hat{Y}=(I-H)Y$, and $RSS=e^{T}e$. The orthogonal projection $I-H$ has rank $n-p$ and annihilates $X\beta$. By Cochran's theorem,Consequently $E({\hat{\sigma }}^2)=(n-p){\sigma }^2/n$, with bias $-p{\sigma }^2/n$. The unbiased estimator isIts square root, the reported residual standard error, estimates $\sigma$; the assertion of unbiasedness applies to the variance, not generally to its square root.

For the paper-strength analysis, let $h_i$ be the measured percentage and $z_i=h_i-\overline{h}$. Both normal linear models use independent errors of common variance ${\sigma }^2$. Their mean functions are ${\beta }_0+{\beta }_1{z}_i$ for lm1, and ${\beta }_0+{\beta }_1{z}_i+{\beta }_2{z}_i^2$ for lm2, with separately fitted coefficients. The reported residual standard errors are $11.82$ and $4.42$, respectively.

For lm1, the estimated conditional expectation at a new percentage $x$ isThe original data mean $\overline{h}$ must be used for centering the new percentage; its numerical value is not supplied in the excerpt. Because ${\sum }_i{z}_i=0$, the two columns of this design matrix are orthogonal, and $\mathrm{Cov}({\hat{\beta }}_0,{\hat{\beta }}_1)=0$. Hence the coefficient standard errors in the output giveThis estimates uncertainty in the mean. Predicting an individual future batch would additionally require the new-error variance, estimated by $11.82^2$.

To compare the nested normal linear models, test $H_0:{\beta }_2=0$ against $H_1:{\beta }_2\ne 0$ within the quadratic model. The Student t-test statistic isIts two-sided p-value is $1.89\times 10^{-8}$. Equivalently $T^2$ is a partial F-test statistic with null distribution $F_{1,16}$. Reject the linear restriction and prefer lm2 at the 5% level. Its residual spread is much smaller; the increase in the coefficient of determination from $0.3054$ to $0.9085$ supports the same conclusion, although the test is the relevant complexity-adjusted comparison.

For regression diagnostics, inspect regression residuals against fitted means and hardwood percentage for omitted curvature, a scale-location plot for nonconstant variance, and a quantile-quantile plot against the normal distribution for departures from the error assumption. Check unusual observations using regression leverage and Cook's distance, and examine residuals in collection or batch order if dependence is plausible. Independence and a common variance require substantive justification as well as these plots; a small p-value for a polynomial term does not itself check the error model.

The command performs a nested-model F-test for $H_0:{\beta }_1=0$ against $H_1:{\beta }_1\ne 0$, retaining the intercept and $x_2$ coefficient in both models. Write $RS{S}_2$ and $RS{S}_3$ for the residual sum of squares in the reduced and full fits. Under $H_0$ and the independent normal-error assumptions,By Cochran's theorem, the numerator and denominator are independent scaled chi-squares with one and $n-3$ degrees of freedom. Equivalently, for this single-coefficient restriction, $F=t_{{\beta }_1}^2$, so its p-value equals that of the two-sided coefficient Student's t-test in model3.

Using the printed $t$ value gives $F=0.800^2=0.640$ and $p\approx 0.43$. The printed estimate and standard error instead giveThis discrepancy is present in the PDF and is too large to be explained by rounding at the shown precision; the underlying ANOVA output is omitted, so an exact unique statistic cannot be recovered. The univariate correlation and $t$ value imply $n\approx 2+2.575^2(1-0.2517297^2)/0.2517297^2\approx 100.006$, consistent with $n=100$ after rounding. With denominator degrees of freedom 97, the two versions give p-values approximately $0.4257$ and $0.4298$, respectively. Either way the partial effect is not significant at 5%.

The sample mean isWrite the Gaussian sample vector as its orthogonal projection onto the span of $(1,\dots ,1)$ plus its projection onto the orthogonal complement. These two Gaussian projections are independent. Their squared standardized lengths give Cochran's theorem:

The hat matrix $P$ is an orthogonal projection of rank $p$, so $I-P$ is an orthogonal projection of rank $n-p$. Because $\epsilon /\sigma$ is a standard multivariate normal distribution vector, Cochran's theorem givesThus ${\hat{\sigma }}^2\stackrel{d}{=}({\sigma }^2/n){\chi }_{n-p}^2$. Substituting this into the preceding formula gives the Distribution of the Akaike information criterion in a normal linear model:

Since $\hat{\beta }$ is an affine transformation of $Y$,The residual projection $I-H$ has rank $n-p$. Cochran's theorem therefore gives

The derivative of the log ratio is $\frac{1}{2}(n/q-1)$, so $\lambda (q)$ increases from zero to one on $(0,n)$ and decreases to zero on $(n,\infty )$. Rejecting for a sufficiently small ratio therefore gives two tails with $0<q_1<n<q_2$.

Under $H_0$, Cochran's theorem gives the exact sampling distribution $Q=S/{\sigma }_0^2\sim {\chi }_{n-1}^2$, independently of the unknown mean. Writing $F_{n-1}$ for its cumulative distribution function, the two cutoffs are determined byThe first equation calibrates the exact size of a statistical test. The second is $\lambda (q_1)=\lambda (q_2)$, so both tails belong to one likelihood ratio cutoff. Hence the exact critical region is $\boxed{\{q\le q_1\}\cup \{q\ge q_2\}}$. Its two tail probabilities are not generally equal: equal-tailed chi-squared cutoffs give a different test.