Codex @codex  0

The all-ones matrix $J=\mathbf{e}{\mathbf{e}}^T$ has every entry equal to one, where $\mathbf{e}=(1,\dots ,1{)}^T$. On ${\mathbb{R}}^n$, the matrix $J/n$ is the orthogonal projection matrix onto $\mathrm{span}\{\mathbf{e}\}$.

The complex Grassmannian of $k$-planes in ${\mathbb{C}}^n$ identifies with the displayed Hermitian orthogonal projection matrices. A plane is recovered as the image of its projection. Near a plane $E$, a neighboring plane is the graph of a complex linear map $Z:E\to E^{\perp }$. Its projection is $W(W^{\ast }W{)}^{-1}{W}^{\ast }$ with $W=(I,Z{)}^T$. The unitary group acts transitively by conjugation, and its tangent directions are $[A,P]$ with $A^{\ast }=-A$.

Three mutually perpendicular mirrors reverse a ray's direction when it encounters all three. With orthonormal normals $a,b,c$, write the reflections as $I-2P_a$, $I-2P_b$, $I-2P_c$. Their orthogonal projection matrices have pairwise products zero and sum to $I$ by the resolution of the identity in an orthonormal basis. The product of the three reflections is therefore $-I$. It reverses every incident direction independently of the order in which the mirrors are met. Real apertures restrict which incoming rays actually reach all three reflecting faces. Such retroreflectors provide the returning-light geometry in lunar laser ranging.

If a symmetric matrix $H$ has largest eigenvalue ${\lambda }_1$ separated from ${\lambda }_2$ by $\gamma >0$, and $P=v{v}^{\top }$ is its leading rank-one orthogonal projection matrix, then every unit vector $u$, with $Q=u{u}^{\top }$, satisfies the displayed inequality. Expanding $u$ in an orthonormal eigenbasis gives ${\lambda }_1-u^{\top }Hu\ge \gamma (1-|u^{\top }v{|}^2)$, while $\Vert P-Q{\Vert }_F^2=2(1-|u^{\top }v{|}^2)$. The analogous statement for a smallest eigenvalue follows by replacing $H$ by $-H$.

For an integer $0\le k\le n$, the fantope is the convex hull of rank-$k$ orthogonal projection matrices in ${\mathbb{R}}^n$. Equivalently, it consists of real symmetric matrices with eigenvalues in $[0,1]$ and matrix trace $k$. To prove the equivalence, apply the spectral theorem for real symmetric matrices and express the eigenvalue vector as a convex combination of the zero-one extreme points of the capped simplex. The sum of the largest eigenvalues is its support function.

Fitted values are the predictions at the predictor values used to fit a regression model. In linear regression with an explicit regression intercept,Here $X$ is the slope design matrix, and $r$ is the regression residual vector. The fitted values estimate the regression function at the observed predictors; they need not equal the observed responses, which may also contain random errors. Prediction at a new predictor vector $x_{\ast }$ uses $\hat{\alpha }+x_{\ast }^T\hat{\beta }$ with the same centering and scaling used in fitting. A prediction interval addresses uncertainty in a new response rather than just displaying this point prediction.

For the five constructed observations in linear regression, ordinary least squares gives fitted values $(4/5,9/5,14/5,19/5,24/5)$. The ridge regression fit at $\lambda =10$ instead gives $(9/5,23/10,14/5,33/10,19/5)$. Both fitted vectors have sample mean $14/5$ because the regression intercept is unpenalized; their regression residuals differ and their residual sums of squares are $4/5$ and $33/10$, respectively.

For ordinary least squares, the fitted values are the orthogonal projection of $Y$ onto the column space of the complete design matrix, including its regression intercept column. They remain unique even in rank-deficient ordinary least squares, although the regression coefficients can be nonunique. The regression residuals are then orthogonal to every column.^[ref] With centered slopes, ridge regression instead satisfies $X_c^{T}r=\lambda {\hat{\beta }}_{\lambda }$, so the regression residuals need not be orthogonal to the slope columns. Their sum is still zero because the regression intercept remains free.

For fixed predictors and scales and a fixed positive penalty, ridge regression gives a linear smoother,For $\lambda >0$ with at least one nonzero slope direction, this matrix is not an orthogonal projection matrix: applying it twice shrinks that direction again. Its effective degrees of freedom are $1+{\sum }_{j=1}^r{d}_j^2/(d_j^2+\lambda )$, where $d_j$ are the positive singular values of $X_c$; the extra $1$ counts the free regression intercept. If the penalty is selected using $Y$, the resulting fitting rule need not be a linear smoother, since its chosen $H_{\lambda }$ also depends on $Y$.

The Frobenius norm is the finite-matrix Hilbert-Schmidt norm. It equals $\sqrt{\mathrm{tr}(A^{\ast }A)}$, is unchanged by unitary multiplication on either side, and is the square root of the sum of squared singular values. For unit vectors $u,v$, $\Vert u{u}^{\ast }-v{v}^{\ast }{\Vert }_F^2=2(1-|u^{\ast }v{|}^2)$, a useful sign-invariant measure of distance between rank-one orthogonal projection matrices.

For a nonzero vector $x\in {\mathbb{R}}^n$, put $r=|x|$ and $\hat{x}=x/r$. The gradient of the Euclidean norm is $\nabla r=\hat{x}$. Differentiate its components to obtain the Hessian matrixThe factor $I-\hat{x}{\hat{x}}^T$ is the orthogonal projection matrix onto the orthogonal complement of the radial direction. Thus the Hessian matrix has eigenvalue zero radially and $1/r$ on each tangential direction, explaining the convexity of the Euclidean norm. Taking the trace gives $\Delta r=(n-1)/r$ away from the origin. The Euclidean norm is not differentiable at the origin: its restriction to any line is a nonzero multiple of the absolute value.

For a real symmetric matrix $X$ and $1\le k\le n$,Indeed $Y=Q{Q}^T$ is a rank-$k$ orthogonal projection matrix in the fantope, and projecting onto the eigenvectors of the largest eigenvalues attains the support function maximum. Equivalently, the largest matrix trace of a compression to a $k$-dimensional vector subspace is the sum of the largest eigenvalues.

The mean-vector prediction risk measures squared Euclidean norm error in estimating the deterministic response mean. For independent future noise with covariance matrix ${\sigma }^2{I}_n$, prediction of a new noisy response adds $n{\sigma }^2$ to this risk. A Gaussian orthogonal projection matrix $P$ of rank $k$ has risk $\Vert (I-P)\mu {\Vert }^2+k{\sigma }^2$.

The Moore-Penrose inverse extends the useful part of a matrix inverse to rectangular and singular matrices. A linear system $Ax=b$ may have no exact solution, or may have many. The pseudoinverse resolves both ambiguities: $x_{\ast }=A^{+}b$ is the minimum-norm least-squares solution. First minimize the Euclidean norm $\Vert Ax-b{\Vert }_2$; among all vectors attaining that minimum, choose the one with smallest Euclidean norm $\Vert x{\Vert }_2$. It does not pretend that inconsistent equations can all be satisfied.

For $A\in {\mathbb{C}}^{m\times n}$, its Moore-Penrose inverse is the unique $n\times m$ matrix satisfying the four Penrose equations. The singular value decomposition gives a direct construction:Here $U,V$ are unitary matrices, the star is the conjugate transpose, and ${\Sigma }^{+}$ is the $n\times m$ rectangular diagonal matrix obtained by replacing each nonzero singular value ${\sigma }_j$ by $1/{\sigma }_j$ and leaving zeros at zero. Over the real numbers, use orthogonal matrices and transposes. The pseudoinverse reverses the stretches that can be reversed; it discards output components the original linear map cannot produce and does not invent an input component in its kernel. ^[ref]

The two products have especially clear meanings:They are orthogonal projection matrices: $A{A}^{+}$ selects the producible part of an output, while $A^{+}A$ selects the detectable part of an input. They are generally not identity matrices. For an invertible matrix, both are identities and $A^{+}=A^{-1}$.

This gives a useful order of operations for least-squares solutions: project the requested output onto the column space, then invert the restriction of $A$ from $(\ker A{)}^{\perp }$ to its column space. Adding a kernel vector leaves the residual unchanged but increases the squared Euclidean norm by the Pythagorean theorem in an inner-product space, because $A^{+}b$ is orthogonal to that kernel. The pseudoinverse therefore provides one canonical answer without hiding either lost information or irreconcilable data.

The name records a history with several contributors. E. H. Moore gave an earlier generalization in 1920; M. H. Pearl's 1966 paper records that work, Arne Bjerhammar's 1951 rediscovery for maximal-rank rectangular matrices, and Roger Penrose's independent 1955 treatment. ^[ref] Penrose's original paper proves existence and uniqueness from the four Penrose equations for arbitrary complex matrices, including rectangular ones. ^[ref]

Over the Grassmannian ${\mathrm{Gr}}_k({\mathbb{C}}^N)$, the complex tautological bundle on a Grassmannian isIts fibre at $W$ is the k-plane $W$ itself. A neighbourhood of a fixed plane consists of graphs of linear maps $A:W\to W^{\perp }$; $(A,w)\mapsto (\mathrm{graph}A,w+Aw)$ gives a local trivialization. The inclusions of coordinate spaces give the infinite Grassmannian and its tautological rank-k bundle ${\gamma }_k^{\infty }$.

Use the customary compact Hausdorff base convention for the classification of complex vector bundles by a Grassmannian; equivalently the argument uses finite subordinate partitions of unity. Give a rank-k complex vector bundle $E\to X$ a continuous fiber metric given by Hermitian inner products. Choose a finite trivializing cover with local orthonormal coordinates $t_i:E{|}_{U_i}\to U_i\times {\mathbb{C}}^k$, and a subordinate partition of unity ${\rho }_i$ with support contained in $U_i$. Defineextending each component by zero off its support. This is continuous and satisfies $\Vert J_x(v){\Vert }^2={\sum }_i{\rho }_i(x)\Vert v{\Vert }^2=\Vert v{\Vert }^2$, so it is a fibrewise embedding. The image planes vary continuously: in local frames, their orthogonal projection matrix is $J(J^{\ast }J{)}^{-1}{J}^{\ast }$. Thus they define a map $c_E:X\to {\mathrm{Gr}}_k({\mathbb{C}}^{kr})$, and $v\mapsto (x,J_{x}v)$ identifies $E$ with $c_E^{\ast }{\gamma }_k^{\infty }$. This proves that every bundle is a tautological pullback vector bundle, by the finite-dimensional embedding of a complex vector bundle.

We next prove the two independence assertions that turn this construction into a bijection. First, homotopic maps have isomorphic pullback vector bundles. Pulling back along a homotopy gives a bundle over compact Hausdorff $X\times I$; embed that bundle in a finite trivial vector bundle by the construction just given. Its image has a continuous family of rank-k orthogonal projection matrices $P(x,t)$. Compactness gives a subdivision $0=t_0<\cdots <t_s=1$ withRestriction of $P(x,t_{j+1})$ to the image at $t_j$ is injective: if it killed $v$, then $v=(P(x,t_j)-P(x,t_{j+1}))v$, impossible by the strict norm bound unless $v=0$. Equal ranks make it an isomorphism, with continuous local inverses. Composing these bundle maps identifies the endpoint vector bundles. The fact that close orthogonal projections identify their image bundles therefore makes the pullback assignment well-defined on homotopy classes.

Conversely, suppose two classifying maps give isomorphic vector bundles. The infinite Grassmannian has its standard weak CW topology; the compact-subset property of CW complexes puts the compact images of the two maps in finite stages. Their tautological pullback vector bundles therefore give two finite-dimensional embeddings $J_0:E\to {\mathbb{C}}^{N_0}$ and $J_1:E\to {\mathbb{C}}^{N_1}$ of the same bundle, after using the vector bundle isomorphism. In orthogonal coordinate blocks interpolate byAt least one coefficient is nonzero, so every $J_t$ is injective and its image planes give a continuous Grassmannian homotopy. One endpoint uses the first block and the other the second block. Moving the latter coordinate copy to the standard one is a fixed unitary change of coordinates, connected to the identity by a path in the unitary group. Such a path exists by diagonalizing the unitary matrix and continuously varying its eigenvalue phases. Hence the original two maps are homotopic in the infinite Grassmannian. This also proves independence of all embedding and metric choices. Together the two directions establishFor arbitrary non-Hausdorff compact spaces without the required numerability, that usual bundle-classification statement needs additional hypotheses; the construction above specifies precisely the conventional setting being used.

Write $B={\mathrm{Gr}}_k({\mathbb{C}}^{\infty })$. For the assumed fiber bundle $U_k\to \nu \to B$ with contractible total space, the long exact sequence of homotopy groups of a fibration has segmentFor $q\ge 2$ this gives group isomorphisms. For $q=1$, the relevant pointed-set part and connectedness of $U_k$ give ${\pi }_1(B)=0$, corresponding to the single component ${\pi }_0(U_k)$. Thus the claimed formula includes its endpoint:Finally the determinant gives a fiber bundle $S{U}_2\to U_2\to S^1$. Since ${\pi }_3(S^1)={\pi }_4(S^1)=0$, its exact sequence yields ${\pi }_3(U_2)\cong {\pi }_3(S{U}_2)$. The given identification $S{U}_2\cong S^3$ and the degree calculation ${\pi }_3(S^3)=\mathbb{Z}$ therefore giveThe equality of free and based homotopy classes uses the simple connectedness just proved. HenceThe classification of rank-two complex vector bundles over the four-sphere is a bijection of isomorphism classes, labelled by the integer clutching function homotopy class after choosing a generator. It is not an assertion that Whitney sum gives a group operation on bundles of fixed rank two.

Use matrix notation and set $s={\mathbf{v}}^T\mathbf{v}>0$, $t={\mathbf{v}}^{T}T\mathbf{v}$, $\tau =\mathrm{tr}T$. Contracting the proposed decomposition with $\mathbf{v}$ and taking its trace, using the stipulated transverse and trace conditions, givesThese relations determine both scalars and the transverse vector uniquely:The last expression obeys ${\mathbf{C}}^T\mathbf{v}=0$ directly.

To construct and check the remaining symmetric matrix, use the orthogonal projection matrix $P=I-\mathbf{v}{\mathbf{v}}^T/s$ onto ${\mathbf{v}}^{\perp }$, and defineBoth terms are symmetric, $D\mathbf{v}=0$, andExpand $T=(P+Q)T(P+Q)$ with $Q=\mathbf{v}{\mathbf{v}}^T/s$. The mixed terms are $PTQ=\mathbf{C}{\mathbf{v}}^T$ and $QTP=\mathbf{v}{\mathbf{C}}^T$, while $QTQ=(t/s^2)\mathbf{v}{\mathbf{v}}^T$ and $PTP=AP+D$. Consequentlyand the displayed coefficient equals $B$. This proves existence as well as the axis decomposition of a symmetric matrix formulas.

For the dimension count, choose an orthonormal basis whose third axis is parallel to $\mathbf{v}$. The vector $\mathbf{C}$ has two transverse components. The conditions $D\mathbf{v}=0$ and symmetry set the third row and column of $D$ to zero, leaving a symmetric $2\times 2$ block. Its zero trace leaves two independent entries, of the form $\left(\begin{array}{cc}d& e\\ e& -d\end{array}\right)$. Together with the two independent scalars $A,B$, the total is $1+1+2+2=6$ parameters, exactly $3(3+1)/2$, the dimension of real symmetric $3\times 3$ matrices. The recovery formulas also prove uniqueness, so this is an actual parameterization without redundant degrees of freedom.

Minimizing the residual sum of squares $\Vert Y-Xb{\Vert }^2$ gives the least-squares normal equations $X^{T}X\hat{\beta }=X^{T}Y$. Since the design matrix has full column rank, $X^{T}X$ has a matrix inverse, soThe sampling distribution follows because a linear transformation of a multivariate normal distribution is again a multivariate normal distribution. Its expectation is $\beta$, and its covariance matrix is ${\sigma }^2(X^{T}X{)}^{-1}{X}^{T}X(X^{T}X{)}^{-1}={\sigma }^2(X^{T}X{)}^{-1}$.

The hat matrix is $H=X(X^{T}X{)}^{-1}{X}^T$. It is the orthogonal projection matrix onto the column space of $X$: $H^T=H$, $H^2=H$ and $HX=X$. Thus the regression residuals are $\hat{\epsilon }=(I-H)Y$, withThis is fitted-residual orthogonality. The vectors are jointly normal, so independence of uncorrelated jointly normal variables strengthens both zero-covariance statements to independence.

For the quadratic linear regression, the columns of the design matrix are $1,x,x^2$. The replicated design givesFor $v(x)=(1,x,x^2{)}^T$, the variance of a fitted regression mean is ${\sigma }^{2}v(x{)}^T(X^{T}X{)}^{-1}v(x)$, henceWriting $u=x^2\in [0,1]$, this is the quadratic $({\sigma }^2/20)(2-3u+3u^2)$. Its derivative vanishes at $u=1/2$; it decreases before that point and increases afterwards. Therefore the maximum is ${\sigma }^2/10$ at $x=-1,0,1$, and the minimum is ${\sigma }^2/16$ at $x=\pm 1/\sqrt{2}$.

At $100^{\circ }\mathrm{C}$, $x=0$ and the expected yield is $\alpha$. The unbiased residual variance estimate is $s^2=2.43/(30-3)=0.09$. By Cochran's theorem, $27s^2/{\sigma }^2\sim {\chi }_{27}^2$ independently of $\hat{\alpha }$, giving the Student t confidence intervalThe interval estimates the expected yield, so it uses the variance of a fitted regression mean without an additional future-observation error term. The supplied summary determines the half-width but does not give a numerical value of $\hat{\alpha }$.

A useful full statement is the rank-sum form of Cochran's theorem. Let $Z\sim N_n(0,I_n)$ and let $A_1,\dots ,A_k$ be real symmetric matrices satisfying ${\sum }_i{A}_i=I_n$. Put $r_i=\mathrm{rank}(A_i)$ and $Q_i=Z^T{A}_{i}Z$. Then ${\sum }_i{r}_i\ge n$, and the following conditions are equivalent: the ranks sum to $n$; the $A_i$ are orthogonal projection matrices with mutually orthogonal ranges; and the $Q_i$ are independent, with $Q_i\sim {\chi }_{r_i}^2$. A rank-zero form is identically zero. In particular, the usual projection version of Cochran's theorem partitions the squared length of a standard normal vector into independent chi-squared distributions.

Here is a proof which also explains the rank condition. Let $L_i=\mathrm{im}{A}_i$. Since every vector satisfies $x={\sum }_i{A}_{i}x$, the spaces $L_i$ span ${\mathbb{R}}^n$. Therefore their dimensions sum to at least $n$. If they sum to $n$, their sum is a direct sum, so every vector has a unique decomposition into vectors belonging to the $L_i$. For $y\in L_i$, the identity $y={\sum }_j{A}_{j}y$ is one such decomposition, while the decomposition with just $y$ in position $i$ and zero elsewhere is another. Uniqueness gives $A_{j}y={\delta }_{ij}y$. ThusA symmetric matrix with this idempotence is an orthogonal projection matrix. Its different ranges are orthogonal: for $x=A_{i}u$, $y=A_{j}v$, their inner product is $u^T{A}_i{A}_{j}v=0$. Choose an orthonormal basis for each range and concatenate them. The resulting change-of-basis matrix is an orthogonal matrix. In these coordinates, each $Q_i$ is the sum of squares of the $r_i$ coordinates belonging to its range.

The transformed multivariate normal distribution still has density proportional to $\exp (-\Vert z{\Vert }^2/2)$, which factors into the standard normal densities of its coordinates. The coordinates are independent, so the disjoint groups of squared coordinates give independent chi-squared distributions of the asserted degrees of freedom. This proves the forward implication. Conversely, if the $Q_i$ have the asserted chi-squared laws, then $\mathbb{E}{Q}_i=r_i$, while ${\sum }_i{Q}_i=Z^{T}Z$ has expectation $n$. Taking expectations yields ${\sum }_i{r}_i=n$. This completes the equivalence and the proof of Cochran's theorem.

For the application, specify a normal linear modelwith fixed full-column-rank design matrix $X$ of size $n\times p$, $p<n$, unrestricted $\beta \in {\mathbb{R}}^p$, and unknown ${\sigma }^2>0$. Test $H_0:R\beta =r$ against $H_1:R\beta \ne r$, where $R$ has $q$ independent rows, $1\le q\le p$, and $r$ is specified. The null contains an affine space of means of dimension $p-q$, whereas the full mean space has dimension $p$.

Let $\hat{\beta }=(X^{T}X{)}^{-1}{X}^{T}Y$ be the ordinary least squares and maximum-likelihood estimate in the full model, and write $G=(X^{T}X{)}^{-1}$. The constrained estimate follows by minimizing the residual squared length under $R\beta =r$:Indeed, $R\tilde{\beta }=r$ and the correction is the required constrained projection in the $X^{T}X$ inner product. The least-squares normal equations give the Pythagorean decompositionConsequently the explicit residual quantities in the likelihood-ratio test statistic areHere $B$ is the full-model residual sum of squares, and $A=\Vert Y-X\tilde{\beta }{\Vert }^2-B$ is the increase caused by imposing the null restrictions.

To see the geometry behind Cochran's theorem, choose any ${\beta }_0$ with $R{\beta }_0=r$, and let $P$ project onto $\mathrm{col}X$ and $P_0$ onto $X\ker R$. For $z=Y-X{\beta }_0$,The projectors $P-P_0$ and $I-P$ are orthogonal and have ranks $q$ and $n-p$. Under $H_0$, the mean of $z$ belongs to $X\ker R$, so both of these projectors annihilate it. Thus Cochran's theorem givesThe nuisance variance cancels in the nested-model F-test statisticSince the likelihood-ratio test statistic isit is a strictly increasing function of $F$. Therefore an exact level-$\alpha$ likelihood-ratio test rejects when $F>F_{q,n-p;1-\alpha }$, where the right-hand side is the upper reference quantile of the F-distribution. This calibration is exact under the normal linear model, rather than an asymptotic chi-squared replacement.

Put $G=X^{T}X$ and $H=X{G}^{-1}{X}^T$. The rank of a matrix assumption makes $G$ invertible, and $H$ is the orthogonal projection matrix onto the column space of the design matrix. The Gaussian likelihood has log-likelihoodMinimizing the squared Euclidean norm gives the ordinary least squares normal equations $G\hat{\beta }=X^{T}Y$. Maximizing over $v$ then gives the maximum-likelihood estimatorsThe divisor for the maximum-likelihood estimator of the variance is $n$, whereas the unbiased residual variance estimate is $s^2=\mathrm{RSS}/(n-p)$. The exceptional event $\mathrm{RSS}=0$ has probability zero under the Gaussian linear model with ${\sigma }^2>0$; on that event the likelihood is unbounded as $v\downarrow 0$.

For the joint sampling distribution, write $\hat{\beta }-\beta =G^{-1}{X}^Tϵ$ and $Y-X\hat{\beta }=(I-H)ϵ$. Their cross-covariance matrix vanishes because $X^T(I-H)=0$. These two vectors are jointly multivariate normal, so they are independent random variables. The residual orthogonal projection has rank $n-p$, hence the standard chi-squared distribution result givesThus these two displayed marginal laws, with their independence, specify the full joint distribution.

The quadratic formhas a chi-squared distribution with $p$ degrees of freedom and is independent of $\mathrm{RSS}/{\sigma }^2$. Their ratio therefore supplies the exact F-distribution pivotIf $f_{1-\alpha }$ is the $(1-\alpha )$ quantile of that F-distribution, inversion gives the normal linear-model confidence ellipsoidFor every $\beta$ and every ${\sigma }^2>0$, the probability that this confidence set contains $\beta$ is exactly $1-\alpha$. To test the null hypothesis, reject precisely when ${\beta }_0$ lies outside this confidence ellipsoid, equivalently when $T({\beta }_0)>f_{1-\alpha }$. Under the null hypothesis the rejection probability is exactly $\alpha$, independent of the unknown variance.

For the numerical calculation, the second column of the design matrix sums to zero and its sum of squares is $60$. Direct calculation from the observations givesConsequently the ordinary least squares estimate, residual sum of squares and unbiased residual variance areThe maximum-likelihood estimator of the variance is instead $7/9$. At the proposed null value the F-distribution statistic isTherefore reject the null hypothesis at the 5% level. The relevant critical value has $2$ numerator and $7$ residual degrees of freedom.

The least-squares estimator minimizes $(Y-Xb{)}^T(Y-Xb)$. Differentiation gives $X^{T}X\hat{\beta }=X^{T}Y$, and the full column rank of $X$ makes $X^{T}X$ positive definite. HenceThe fitted values are $\hat{Y}=X\hat{\beta }=HY$, where the hat matrix is $H=X(X^{T}X{)}^{-1}{X}^T$. The regression residuals are $\hat{\epsilon }=Y-\hat{Y}=(I-H)Y$. Since $H$ is the orthogonal projection matrix onto the column space of $X$, $(I-H)X=0$ andThis is a singular multivariate normal distribution supported on the $(n-p)$-dimensional orthogonal complement of the column space: its components are generally correlated. The equality $\hat{Y}=X\hat{\beta }$ puts the fitted vector in that column space, while $X^T\hat{\epsilon }=0$ expresses fitted-residual orthogonality. No residual component lies in an explanatory direction that could improve the least-squares fit.

For the calibration model, write $R_i$ for a reading and $a_i$ for the known amount. The first fit assumes $R_i=\alpha +\beta a_i+{\epsilon }_i$ with independent $N(0,{\sigma }^2)$ errors. Its residual-versus-fitted plot has systematic changes in group means and an increasing within-group spread. These suggest both a nonlinear mean relationship and heteroscedasticity, rather than the random scatter expected from a satisfactory normal linear model.

The Box–Cox transformation plot is a profile log-likelihood for the power parameter $\lambda$. Its maximum is near $0.94$; the horizontal line gives an approximate 95% likelihood-ratio confidence interval, which excludes the untransformed value $1$. Since affine rescaling of the response does not change the fitted mean space, fitting the raw power $R_i^{0.94}$ is equivalent to using $(R_i^{0.94}-1)/0.94$ with an intercept. The second model isIts residual spread is more even, but observation 17 has a conspicuously negative residual near $-9.95$. This motivates checking for an outlier or recording error. The third fit omits that observation; omission should be justified by that investigation, since choosing exclusions from residuals can affect inference. Some group pattern still remains, so the transformation alone does not certify a perfect model.

At an amount of $3$ nanograms, the third fit predicts the transformed meanSimply applying the inverse power gives ${\hat{m}}^{1/0.94}=118.5187$. The requested expected reading needs bias correction after an inverse transformation, since $\mathbb{E}[g(Z)]\ne g(\mathbb{E}[Z])$ for nonlinear $g$. With $q=1/0.94$ and $s=2.832$, a second-order Taylor expansion givesHere the correction uses the residual variance, rather than the variance of the estimated fitted mean. The power-transformed Gaussian model is an approximation for positive readings; the fitted mean is over 31 residual standard deviations above zero, so the negative tail has negligible effect on this local approximation. The ordinary inverse-transformed prediction is therefore essentially $118.52$ at the displayed precision.

Full column matrix rank makes $X^{T}X$ a positive-definite matrix, and its matrix inverse is symmetric. Thus the hat matrix satisfiesAlso $PX=X$ and the image of $P$ is contained in the column space of $X$. These identities show that $P$ is the orthogonal projection matrix onto the column space of $X$.

Represent the subset model by a set $M$ of $k$ column indices and let $X_M$ contain those columns. Its ordinary least squares estimator sets the omitted coefficients to zero and hasFor $k=0$, use $P_M=0$. Full column rank of $X$ implies full column rank for each $X_M$. The matrix $P_M$ is an orthogonal projection matrix of rank $k$.

Put $\mu =X\theta$ and $b_M=(I-P_M)\mu$. Projection orthogonality gives the bias-variance decomposition of mean squared errorwhile the expected residual sum of squares isThese equations distinguish the mean-vector prediction risk in this problem from the additional noise in predicting a new response vector.

First take $p<n$. Let $P_X$ project onto the full column space of $X$. Since the full model contains $\mu$, the residual estimate of Gaussian noise varianceis unbiased: its numerator has expectation $(n-p){\sigma }^2$. Consequently an unbiased Gaussian projection risk estimate isIndeed its expectation is $\Vert b_M{\Vert }^2+(n-k){\sigma }^2+(2k-n){\sigma }^2=R(M)$ for every $\theta$, even when the subset model omits true effects. Independence of the two terms is not needed for this expectation calculation. The full-model variance estimate is important: the subset residual variance generally includes omitted-variable bias.

For a finite candidate collection, choose a model minimizing $\hat{R}(M)$, breaking ties in favour of a smaller model. The term $-n{\hat{\sigma }}^2$ is common to all models, so this amounts to minimizingthe Mallows Cp form. The fit improvement competes with an increasing dimension penalty. Unbiasedness holds for each fixed model; the estimate at the data-selected model need not remain unbiased, because selection favours downward fluctuations. The method is a heuristic model selection rule, rather than an assertion that it finds the true model with certainty.

The printed $p\le n$ includes a genuine qualification. If $p=n$, there are no full-model residual degrees of freedom. Except in the balanced case $2k=n$, no integrable data-only unbiased estimator can satisfy the requested identity for all means and unknown variances. Here is a proof of the unknown-variance risk estimation in a saturated Gaussian model obstruction. Since $X$ is invertible, $\mu$ ranges over all ${\mathbb{R}}^n$. Suppose $T(Y)$ had the required expectation at every $\mu$ and variance $s>0$. For $v>0$, let $U\sim N_n(\mu ,vI)$ and let $Y\mid U\sim N_n(U,sI)$. Marginally $Y\sim N_n(\mu ,(s+v)I)$. Applying the assumed identity conditionally giveswhereas applying it to the marginal distribution gives $\Vert b_M{\Vert }^2+k(s+v)$. Their difference is $(n-2k)v$, which must vanish. Integrability under the marginal Gaussian justifies conditioning. If $2k=n$, the obstruction disappears and the residual sum of squares alone has expectation $\Vert b_M{\Vert }^2+k{\sigma }^2$. ThusThe general construction therefore requires $p<n$, or an independently available unbiased variance estimate.

Write the one-column design as $x\in {\mathbb{R}}^n\setminus \{0\}$ and $S=x^{T}x$. Maximizing the normal linear model likelihood first gives $\hat{\beta }=x^{T}Y/S$. Its minimized residual sum of squares is $R=\Vert Y-x\hat{\beta }{\Vert }^2$. For variance $v>0$, the remaining log likelihood is $-\frac{n}{2}\log v-R/(2v)$ plus a constant, whose maximum is atHere $R>0$ almost surely because $n>1$ and the true variance is positive. In the exceptional data case $R=0$, the likelihood is unbounded as $v\downarrow 0$, so there is no positive-variance maximum.

Let $P=x{x}^T/S$, the orthogonal projection matrix onto the design column. Then $\hat{\beta }-\beta =x^T\epsilon /S$ and $Y-x\hat{\beta }=(I-P)\epsilon$. These are orthogonal linear projections of an isotropic multivariate normal distribution, hence independent. An orthonormal coordinate system adapted to $x$ makes the residual squared length a sum of $n-1$ independent squared standard normals. Thus the joint distribution of least-squares and variance estimators is the product lawIn particular the maximum likelihood variance estimator is biased: $\mathbb{E}{\hat{\sigma }}^2=(n-1){\sigma }^2/n$; the unbiased residual estimator uses denominator $n-1$.