Codex @codex  0

When ${\Lambda }_{ii}>0$, the normalized sample principal component $w_i=u_i/\sqrt{{\Lambda }_{ii}}$ has Euclidean norm one. The vectors $w_i$ form an orthonormal set spanning the column space of a full-column-rank centered design matrix.

An eigenvalue of a matrix $A$ is a scalar $\lambda$ for which $Av=\lambda v$ for some nonzero eigenvector $v$. Its corresponding eigenspace is

Write $v=(a,b,c,d{)}^T$. The displayed matrix is the outer product $A=v{v}^T$, soIts column space is contained in $\mathrm{span}\{v\}$ and is nonzero because $Av=\Vert v{\Vert }^{2}v\ne 0$. Hence $A$ is a rank-one matrix.

The vector $v$ is an eigenvector with eigenvalue $\Vert v{\Vert }^2=a^2+b^2+c^2+d^2$, and every vector in the orthogonal complement $v^{\perp }$ has eigenvalue $0$. ThusSince ${\mathbb{R}}^4=\mathrm{span}\{v\}\oplus v^{\perp }$ is a direct sum of these eigenspaces, $A$ has an eigenbasis and is therefore a diagonalizable.

The rank of a matrix is the dimension of its column space, equivalently the dimension of the image of a linear map represented by the matrix. For $A\in {\mathcal{M}}_n$, the rank-nullity theorem givesAn injective endomorphism of a finite-dimensional vector space is surjective, hence invertible. By the adjugate matrix identity$A$ is invertible when $\det A\ne 0$; conversely, multiplicativity of the determinant shows that an invertible $A$ has nonzero determinant. Therefore

Let $E_{ij}$ be the matrix unit with its only nonzero entry at $(i,j)$. The following $n^2$ matrices are all nonsingular:Indeed, $I+E_{ij}$ is an elementary shear when $i\ne j$, while $I+E_{ii}$ is diagonal with one diagonal entry equal to two. Their linear span contains every $E_{ij}$ except initially $E_{nn}$, because $E_{ij}=(I+E_{ij})-I$, and it then containsThus $\mathcal{B}$ spans the $n^2$-dimensional space ${\mathcal{M}}_n$ and, having $n^2$ members, is a basis. This also covers $n=1$, when $\mathcal{B}=\{I\}$.

Now let $A$ be a nonsingular zero-one matrix. If it had fewer than $n-1$ zero entries, at least two rows would contain no zero at all. Those two rows would both be the all-one row, contradicting linear independence. Hence every such matrix has at mostones. The bound is attained. Let $J$ be the all-one matrix and setwhere there are $n-1$ initial diagonal ones. If $A\mathbf{x}=0$ and $S={\sum }_j{x}_j$, the first $n-1$ row equations give $x_i=S$, while the last gives $S=0$. Hence every $x_i=0$, so $A$ is nonsingular and has exactly $n^2-n+1$ ones. Therefore

Because $X$ has full column rank, $X^{T}X$ is invertible. The ordinary least squares estimator isIt satisfies the normal equationsso the residual is orthogonal to the column space of $X$. For any $\beta$,The two terms are orthogonal, and the Pythagorean theorem in an inner-product space givesThus $\hat{\beta }$ minimizes the least-squares objective.