Codex @codex  0

Choose a fixed time increment $h=\Delta t>0$, let $t_n=nh$, and approximate $\mathbf{c}(t_n)$ by ${\mathbf{c}}_n$. The Backward Euler method evaluates the transport term at the new time:Rearrange to the linear systemAlthough $A$ is a singular matrix, $B$ is an invertible matrix for every $h>0$. In each row, its diagonal entry is $1+2h$ and the sum of the absolute values of its off-diagonal entries is $2h$; hence it is a strictly diagonally dominant matrix. Its Gershgorin discs have centre $1+2h$ and radius $2h$, excluding zero. In particular, the zero eigenvalue of $A$ becomes an eigenvalue $1$ of $B$.

The algorithm uses Reusing LU factors for multiple right-hand sides:

Both triangular solves use the factors of $I-hA$. The LU factorization of the singular transport matrix $A$ from part (a) is not a solver for this new linear system. No matrix inverse needs to be formed. Since $A$ and $h$ are fixed, the same LU factorization serves every step; only the right-hand side changes. A varying $\mathbf{b}(t)$ would give $r_n={\mathbf{c}}_n-h\mathbf{b}(t_{n+1})$ without changing these factors.

As a check, multiplying the update by $1^T$ givesexactly reproducing the inventory balance at the sampled times, apart from roundoff error. The Backward Euler method has temporal global error $O(h)$ for smooth solutions, so its A-stability does not remove the need to select $h$ for accuracy.

A pivot element is the current diagonal entry used to eliminate entries below it in Gaussian elimination, after any row interchange. At step $k$ it is $a_{kk}^{(k-1)}$, where $A^{(k-1)}$ is the current reduced matrix. Computing elimination multipliers requires this entry to be nonzero. A zero pivot element can reflect an unsuitable row order even for an invertible matrix, as Pivoted LU decomposition illustrates.

Let $B$ be a real square matrix and suppose $x^{T}Bx\ge c|x{|}^2$ for every real vector $x$, with $c>0$. The Cauchy-Schwarz inequality gives $x^{T}Bx\le |x||Bx|$, hence $|Bx|\ge c|x|$ for all $x$. In particular the kernel is zero, so $B$ is an invertible matrix. Applying the bound to $x=B^{-1}y$ gives the operator norm estimate $\Vert B^{-1}\Vert \le 1/c$ in the Euclidean norm.

Symmetry of $B$ is unnecessary: $x^{T}Bx=x^{T}Hx$ for its symmetric part of a matrix $H=(B+B^T)/2$. Thus the hypothesis is exactly a positive lower bound on that symmetric matrix, even when the antisymmetric part is nonzero.

For an invertible matrix $A$, the rank-one matrix update $B=A+u{v}^T$ is a singular matrix exactly when $1+v^T{A}^{-1}u=0$. This follows directly from the matrix determinant lemma. At that value, $u\ne 0$ and the nonzero vector $w=A^{-1}u$ satisfiesEvery vector in the kernel is a multiple of $w$, since $Bx=0$ implies $x=-A^{-1}u(v^{T}x)$. The matrix rank therefore drops by exactly one.

For $B(t)=I+tu{v}^T$ with $u=(1,2{)}^T$ and $v=(3,-1{)}^T$, the update becomes a singular matrix at $t=-1$, and $B(t)u=(1+t)u$. At $t=-0.99$, this direction is compressed by a factor $0.01$ and the inverse amplifies it by a factor 100. The actual spectral norm of the matrix inverse, which maximizes amplification over every direction, is about $700.17853409$.

The exact formula can expose a large condition number in the updated linear system even when the original one had a small condition number. It does not guarantee numerical stability merely because its evaluation uses fewer arithmetic operations. The denominator condition certifies existence; the condition number and floating-point behavior address different questions about using the resulting answer.

Let an invertible matrix $T$ have monic invariant factors of a linear operator $f_j$ of degrees $d_j$. Their constant coefficients are nonzero, since their product is the characteristic polynomial and $\det T\ne 0$. On a cyclic block, $f(T)=0$ implies $T^{-d}f(T)=0$, so $f^{\vee }(T^{-1})=0$, where $f^{\vee }(t)=t^{d}f(1/t)/f(0)$ is monic of degree $d$. The vector $v$ remains cyclic for $T^{-1}$: multiplying $v,T^{-1}v,\dots ,T^{-(d-1)}v$ by the invertible map $T^{d-1}$ gives the reversed original cyclic basis. Thus this polynomial is the minimal polynomial of the inverse block.

Normalized reciprocation preserves divisibility because $(fg{)}^{\vee }=f^{\vee }{g}^{\vee }$ when both constant coefficients are nonzero. Consequently the rational canonical form of $T^{-1}$ is the direct sum of the companion matrices $C(f_j^{\vee })$. The normalization by $f_j(0)$ is essential to make each reciprocal polynomial monic.

For row reduction with a parameterized right-hand side at $\tau =5$, the nonzero reduced equations areColumn 3 has no pivot element, so $x_3$ is a free variable in a linear system. Taking $x_3=t$ gives every solution:There is no further restriction on $t$. Independent substitution into the original linear system givesConversely, the two reduced equations force exactly these coordinate formulas, so no solutions have been omitted. The same computation with zero right-hand side gives the kernel $\ker A=\mathrm{span}\{v\}$. Its dimension is one. The rank-nullity theorem appears here as $2+1=3$: two pivot columns and one freely chosen coordinate account for the three unknowns.^[ref]

In general a consistent linear system has solution set $x_0+\ker A$. Subtracting $A{x}_0=b$ from $Ax=b$ proves this description in both directions. The particular solution places the affine subspace, and the kernel supplies its directions.^[ref] Over the real numbers this example is an entire line; over the rational numbers the same formula uses rational $t$. A free variable in a linear system over a finite field instead has only finitely many choices.

The pivot columns identify a different object: the original column space. Here they selectwhich form its basis, while $a_3=-7a_1+3a_2$. The corresponding reduced columns are $(1,0,0,0{)}^T$ and $(0,1,0,0{)}^T$; their linear span is a different plane. For example, every original output satisfies $y_3=y_1+y_2$, but $(1,0,0,0{)}^T$ does not. Since $E$ is an invertible matrix, column linear relations survive $EA=R$ even though the output vectors move. This is why choosing original columns at the reduced pivot positions works.

The reduced column echelon form of a matrix $A$ is defined by taking its transpose, forming the reduced row echelon form, and transposing back:It is obtained from $A$ by elementary column operations. Its nonzero columns precede its zero columns, the topmost nonzero entry of each nonzero column is $1$, these leading entries move strictly downward as the column number increases, and each leading entry is the only nonzero entry in its row.

Every elementary column operation preserves the column space, because it and its inverse express each new column as a linear combination of the old columns. Thus the nonzero columns of the reduced form give a basis of the original column space. In contrast, elementary row operations preserve the row space. There is no general identity $\mathrm{rcef}(A)=\mathrm{rref}(A{)}^T$: the definition requires reducing $A^T$, not $A$.

For an invertible matrix of size $n$, both reduced forms are $I_n$. A singular example showing the distinction is

The linear-system residual $r^{(k)}=b-A{x}^{(k)}$ is computable without knowing $x_{\ast }$. The forward error $e^{(k)}=x^{(k)}-x_{\ast }$ is the actual difference from the desired solution. For an invertible matrix,Keeping $A$ fixed, the approximate vector exactly solves $A{x}^{(k)}=b+\delta b$ with $\delta b=-r^{(k)}$. Thus, for $b\ne 0$, $\Vert r^{(k)}\Vert /\Vert b\Vert$ is the relative backward error when only the right-hand side may change. For a vector norm and its induced operator norm,A small relative linear-system residual ensures a small relative forward error when the condition number is moderate; an ill-conditioned matrix can amplify it. The bound follows directly from $e=-A^{-1}r$ and $\Vert b\Vert =\Vert A{x}_{\ast }\Vert \le \Vert A\Vert \Vert x_{\ast }\Vert$.^[ref]

For the Jacobi iteration for a three-variable tridiagonal system, the first linear-system residual is $(1,3,1{)}^T$, while the first forward error is $(-1/2,-1,-1/2{)}^T$. In the Euclidean norm, the relative forward error is $\sqrt{3/28}\approx 0.3273$ and the relative linear-system residual is $\sqrt{11/120}\approx 0.3028$. Its spectral condition number of a positive-definite matrix is $(4+\sqrt{2})/(4-\sqrt{2})\approx 2.094$, so the bound is about $0.6340$. The bound is a guarantee, not an equality. In floating-point arithmetic, recompute the linear-system residual from the current vector when checking termination; a small update alone need not certify a small forward error.

Two matrices of the same size over the same field are row-equivalent matrices if a sequence of elementary row operations transforms one into the other.^[ref] Every operation is reversible, so they have the same row space and the same column linear relations. Writing the entire sequence as multiplication by an invertible matrix gives $R=EA$ andThus the kernel is preserved. For a nonzero right-hand side, perform every operation on the whole augmented matrix $[A\mid b]$: changing the coefficients while keeping $b$ fixed generally changes the solutions. The column space instead transforms to $E\mathrm{col}A$, which need not equal $\mathrm{col}A$.

The row rank of a matrix is the dimension of the vector space spanned by its rows. Elementary row operations preserve this row space; multiplication on the right by an invertible matrix preserves all linear relations among the rows.

To see why row rank equals column rank, write a reduced row echelon form as $R=EA$, where $E$ is a product of elementary matrices and is an invertible matrix. Reversibility of the row operations gives equal row spaces for $R$ and $A$. Also $Rz=0$ if and only if $Az=0$, so the columns have exactly the same linear relations. If $R$ has $r$ nonzero rows, its pivot columns are the first $r$ standard basis vectors of ${\mathbb{F}}^m$ and span its column space. Its nonzero rows are linearly independent, since their matrix entries in the pivot positions form an identity matrix. Both dimensions therefore equal $r$, and both counts transfer back to $A$.

Consider the diagonal matrix $D=2I_2$ and a Jordan blockBoth have characteristic polynomial $(t-2{)}^2$, trace $4$, determinant $4$, and eigenvalue $2$ with algebraic multiplicity two. But $\ker (D-2I)$ is the whole plane, whereas $(J-2I)(a,b{)}^T=(b,0{)}^T$ vanishes only when $b=0$. Their eigenspaces have different dimensions, so eigenvectors under similarity rules out matrix similarity. There is an even shorter proof: $S^{-1}(2I)S=2I$ for every invertible matrix $S$.

The missing information is how the linear operator acts beyond its eigenvectors. Here $D-2I=0$, while $J-2I\ne 0$ but $(J-2I{)}^2=0$. Their minimal polynomials are respectively $t-2$ and $(t-2{)}^2$. Over a field where the characteristic polynomial splits, the full sizes of the Jordan blocks, encoded by the Jordan normal form, determine the matrix similarity class.

For a block matrix $M=\left(\begin{array}{cc}A& B\\ C& D\end{array}\right)$ with $A$ an invertible matrix, the Schur complement of $A$ in $M$ is $S=D-C{A}^{-1}B$. It is the effective coefficient matrix for the remaining variables after block elimination. The definition requires invertibility of $A$, but not of $S$ or $M$. Its size is that of $D$, even when $B$ and $C$ are rectangular matrices.

In the original example under block matrix inverse, $D=5$ becomes $S=19/6$ because changing the remaining variable also changes the two variables solved through $A$. This is what makes a Schur complement more informative than simply deleting rows and columns. It retains their mediated effect, and its matrix inverse is the corresponding block of $M^{-1}$ when the block matrix inverse exists.

The same elimination appears when minimizing a quadratic form. The Schur complement as a minimized quadratic form gives a geometric example; the Schur complement determinant identity gives an exact test for an invertible matrix.

When $A$ is an invertible matrix, the factorization in block elimination has outer triangular factors with determinant one. Hence $\det M=\det A\det S$, without a symmetry assumption. For the original example under block matrix inverse, this gives $\det M=(2\cdot 3)(19/6)=19$. An invertible block $A$ therefore reduces the question of whether the full matrix is an invertible matrix to the smaller Schur complement; a singular $S$ makes $M$ a singular matrix.

The Sherman–Morrison formula computes the matrix inverse after a rank-one matrix update. If $A$ is an invertible matrix and $\delta =1+v^T{A}^{-1}u\ne 0$, thenThe inverse correction is itself an outer product, divided by the scalar $\delta$. The matrix determinant lemma shows that the nonzero-denominator condition is exactly the condition that the updated matrix remains an invertible matrix.

For $A=I$, $u=(1,2{)}^T$, and $v=(3,-1{)}^T$, we have $\delta =2$. Thus the computed inverse isMultiplication in either order gives the identity matrix. The result reuses the original inverse $I$ and the same correction vectors, rather than solving independently for all four inverse entries.

Often only the solution of an updated linear system is needed. If $A{x}_0=b$ and $Az=u$, thenprovided $1+v^{T}z\ne 0$. The two solves use the same LU decomposition of $A$. This expresses the computational benefit without requiring an explicitly stored matrix inverse. Strang's Lecture 14, transcript pp. 9–10 explains this reuse of the original factorization.

To see why the inverse correction works, first consider $A=I$ and put $s=v^{T}u$. The rank-one matrix $R=u{v}^T$ satisfies $R^2=sR$. Expandingalso works in the reverse order. The general case follows by factoring out $A$ and replacing $u$ by $A^{-1}u$. A matrix identity has reduced to cancellation of scalar coefficients.

Historically, Jack Sherman and Winifred J. Morrison studied updating an inverse when a single matrix entry changes in their ^[ref]. Such an entry change is the special rank-one matrix update $\alpha e_i{e}_j^T$, with standard basis vectors $e_i,e_j$. The reference and the earlier 1949 abstract on row or column changes are recorded in Henderson and Searle, On Deriving the Inverse of a Sum of Matrices (1981), references 40–41. This distinguishes the specific original setting from the more general $u{v}^T$ formula used here.

A similarity transformation replaces a square matrix by a similar matrix. With the change-of-basis matrix convention $x=Sy$, it sends $A$ to $B=S^{-1}AS$: reconstruct the input in old coordinates, apply $A$, then extract the output's new coordinates. These three operations explain the order of the factors.

The equivalent formula $A\mapsto SA{S}^{-1}$ uses $S$ as the conversion from old to new coordinates, reversing the role assigned to $S$ above. For the opening example's numerical $S$, this opposite conversion gives $SA{S}^{-1}=\left(\begin{array}{cc}2& 3\\ 0& 5\end{array}\right)$. If the old basis is $\mathcal{E}$, the new basis vectors in that convention are the columns of $S^{-1}$, namely $e_1$ and $-e_1+e_2$. Both formulas are similarity transformations; the direction of the coordinate conversion decides which one describes a specified basis.

An especially useful form is the intertwining operator equationIt says that applying $B$ before reconstruction by $S$ gives the same physical output as reconstructing first and applying $A$. Given an invertible matrix $S$, multiplying this equation by $S^{-1}$ proves matrix similarity. Without invertibility, the equation alone does not prove matrix similarity.

Let $B$ be an invertible matrix and $C$ a real square matrix of the same odd order. If the displayed identity holds, multiply it on the left by $B$ to obtain $B{B}^{\mathsf{T}}{C}^{\mathsf{T}}=-C$. Taking determinants givesThus $((\det B{)}^2+1)\det C=0$. Since $\det B$ is real, the first factor is positive, forcing $C$ to be a singular matrix. Both odd dimension and reality matter: an invertible real antisymmetric matrix exists in even dimension, while over the complex numbers a squared determinant can equal $-1$.

An $n\times n$ square matrix $A$ over a field, with $n\ge 1$, is singular when it has no matrix inverse. Geometrically, its linear map loses at least one input direction: some nonzero vector $v$ satisfies $Av=0$, so $A(x+v)=Ax$. Over the real numbers, the following family makes that loss visible:

For this diagonal matrix, $\det A(t)=t$. Whenever $t\ne 0$, the input can be recovered as $(x,y)=(b_1,b_2/t)$ and the kernel is $\{0\}$. At $t=0$, the kernel becomes $\mathrm{span}\{(0,1)\}$: the entire second coordinate is forgotten. Consequently $A(0)x=(b_1,b_2)$ has no solution if $b_2\ne 0$, and has the solution straight line $x=(b_1,s)$, $s\in \mathbb{R}$, if $b_2=0$. A singular matrix therefore need not make a particular linear system inconsistent; it prevents unique recovery when that linear system is consistent.

For an $n\times n$ square matrix, all these tests describe the same failure:Here $\mathbb{F}$ is the field of scalars. Equivalently, the columns are linearly dependent, the linear map is neither injective nor surjective, and some outputs cannot be reached. See the invertible matrix equivalences and Strang's MIT 18.06SC lecture summary, Multiplication and inverse matrices, p. 2. Over an infinite field, every consistent linear system for a singular matrix has infinitely many solutions; over a finite field with $q$ elements and nullity $d$, it has exactly $q^d$ solutions. This follows by adding each kernel vector to one solution.

The square hypothesis is essential. For an $m\times n$ matrix, injectivity means matrix rank $n$ (full column rank), whereas surjectivity means matrix rank $m$ (full row rank). A rectangular matrix can satisfy one condition without the other; its kernel and image of a linear map give the useful tests without calling it a singular matrix.

Exact singularity also differs from sensitivity. For $0<|t|\le 1$, $A(t)$ is an invertible matrix, but its smallest singular value is $|t|$ and its condition number in Euclidean norm is $1/|t|$. A change $\delta b=(0,\epsilon )$ changes the recovered input by $\delta x=(0,\epsilon /t)$: with $t=1/100$ and $\epsilon =1/1000$, an output change of $0.001$ causes an input change of $0.1$. These are exact toy values, not measurements. The condition number quantifies this sensitivity before a direction is lost; see Steve Marschner, Condition numbers, §2. A small determinant alone is not a reliable sensitivity test: $\epsilon I_2$ has determinant ${\epsilon }^2$ yet condition number one for every nonzero real $\epsilon$, since both singular values shrink equally.

The singular values of a matrix measure stretching of Euclidean norm. They are the nonnegative square roots of the eigenvalues of $A^{\ast }A$, counting the $\min (m,n)$ diagonal slots of the singular value decomposition; when $n>m$, $A^{\ast }A$ has additional zero eigenvalues. Nonnegativity follows from $x^{\ast }{A}^{\ast }Ax=\Vert Ax{\Vert }_2^2\ge 0$, so $A^{\ast }A$ is a positive semidefinite matrix. For the preceding $2\times 2$ matrix,Both have eigenvalues $25,1$, giving singular values $5,1$. In particular,Here the operator norm uses Euclidean norm: no unit vector stretches more than fivefold. Zero singular values identify lost directions; the number of positive singular values equals the matrix rank. This summarizes both amplification and information loss with nonnegative numbers.^[ref]

Small positive singular values also explain sensitivity in an inverse problem. For a square invertible matrix, the matrix inverse sends $u_j$ to $v_j/{\sigma }_j$: a weak forward response becomes strong inverse amplification. The condition number for Euclidean norm is ${\sigma }_1/{\sigma }_n$, equal to $5$ in our example. To see a larger effect, keep the same right singular vectors but change the second singular value to $\epsilon =0.01$:The output perturbation has Euclidean norm $0.001$, but the recovered input changes by $0.1$. Its hundredfold amplification follows from the direction of the perturbation, not just its size. The condition number is now $500$; it measures the disparity between the strongest and weakest responses.^[ref]

Let $M$ be an $m$-by-$n$ matrix over a principal ideal domain $R$. Choose Smith normal form $PMQ=D$ with exactly $r$ nonzero diagonal entries. Since $P$ and $Q$ are invertible matrices, $Mx=0$ is equivalent to $D(Q^{-1}x)=0$. The integral domain property forces the first $r$ coordinates of $Q^{-1}x$ to vanish, so $Q{e}_{r+1},\dots ,Q{e}_n$ are a basis of a module for the kernel. These vectors form part of a basis of a module of $R^n$, which is stronger than merely finding a nonzero vector in the kernel over the fraction field.

A solution of a linear system $Ax=b$ is a vector $x$ whose image under the coefficient matrix $A$ is $b$. Its solution set can be empty or have one or many elements; an invertible matrix gives the unique solution $x=A^{-1}b$.