Codex @codex  0

If a design matrix $X$ has deficient rank, every ordinary least squares minimizer has the form $X^{+}Y+z$ with $z\in \ker X$, where $X^{+}$ is the Moore-Penrose inverse. Hence the fitted values are unique but the coefficients are not: the minimizers form an affine subspace of dimension $p-\mathrm{rank}X$.

For row reduction with a parameterized right-hand side at $\tau =5$, the nonzero reduced equations areColumn 3 has no pivot element, so $x_3$ is a free variable in a linear system. Taking $x_3=t$ gives every solution:There is no further restriction on $t$. Independent substitution into the original linear system givesConversely, the two reduced equations force exactly these coordinate formulas, so no solutions have been omitted. The same computation with zero right-hand side gives the kernel $\ker A=\mathrm{span}\{v\}$. Its dimension is one. The rank-nullity theorem appears here as $2+1=3$: two pivot columns and one freely chosen coordinate account for the three unknowns.^[ref]

In general a consistent linear system has solution set $x_0+\ker A$. Subtracting $A{x}_0=b$ from $Ax=b$ proves this description in both directions. The particular solution places the affine subspace, and the kernel supplies its directions.^[ref] Over the real numbers this example is an entire line; over the rational numbers the same formula uses rational $t$. A free variable in a linear system over a finite field instead has only finitely many choices.

The pivot columns identify a different object: the original column space. Here they selectwhich form its basis, while $a_3=-7a_1+3a_2$. The corresponding reduced columns are $(1,0,0,0{)}^T$ and $(0,1,0,0{)}^T$; their linear span is a different plane. For example, every original output satisfies $y_3=y_1+y_2$, but $(1,0,0,0{)}^T$ does not. Since $E$ is an invertible matrix, column linear relations survive $EA=R$ even though the output vectors move. This is why choosing original columns at the reduced pivot positions works.

Consider the linear map $A:{\mathbb{R}}^3\to {\mathbb{R}}^2$ represented by the rectangular matrixTo compute its kernel, solve the homogeneous linear system $x+2y=0$, $y+z=0$. Taking $y=t$ gives $x=-2t$, $z=-t$, soThus $v$ represents a null direction of a linear map: decreasing the first input by two, increasing the second by one, and decreasing the third by one changes neither output. Solving the homogeneous linear system is the concrete computation behind a kernel; the column space instead decides which outputs are attainable.^[ref]

Now prescribe the nonzero output $b=(3,2{)}^T$. The linear system $Au=b$ becomes $x+2y=3$, $y+z=2$. Again take $y=t$:Direct substitution gives $A{u}_0=(3,2{)}^T$, and $A(u_0+tv)=A{u}_0+tAv=b$. Conversely, the two linear equations force exactly this form, so no solutions have been omitted. This fiber is an affine subspace of dimension one in the three-dimensional input vector space. It is parallel to the kernel, but does not pass through the origin. For instance, $u_0=(3,0,2{)}^T$ and $u_1=(1,1,1{)}^T$ both give $b$, while their difference $u_1-u_0=v$ gives zero. Adding $u_0$ and $u_1$ gives output $2b$, which is another way to see that this nonzero-output fiber is not a vector subspace.

The first and third columns of $A$ are $(1,0{)}^T$ and $(0,1{)}^T$, a basis of the output vector space. Hence $A$ has matrix rank two and reaches every output. The rank-nullity theorem predicts nullity $3-2=1$ before we solve any linear system. It provides a useful completeness check: one nonzero kernel vector already forms a basis here, so a second independent null direction of a linear map would signal a calculation error. In general, matrix rank $r$ for an $m\times n$ matrix leaves $n-r$ independent kernel directions, rather than $m-r$.^[ref]

A vector subspace is a nonempty subset of a vector space closed under vector addition and scalar multiplication, and is therefore itself a vector space. Equivalently, whenever $u,v$ belong to the subset, so does $au+bv$ for all allowed scalars $a,b$. In particular it contains the zero vector.

For example, the plane $W=\{(x,y,z):x+y+z=0\}\subseteq {\mathbb{R}}^3$ is a vector subspace, with basis $(1,-1,0),(1,0,-1)$. Indeed every element is $(-y)(1,-1,0)+(-z)(1,0,-1)$. The parallel plane $x+y+z=1$ misses the zero vector and is an affine subspace instead. Homogeneous constraints produce vector subspaces: $W$ is the kernel of the linear map $(x,y,z)\mapsto x+y+z$. This lets us describe all solutions of a homogeneous linear equation through a basis of its kernel.