Codex @codex  0

A Runge-Kutta method advances $y^{\prime }=f(t,y)$ through stagesThe Butcher tableau records $A,b,c$, normally with $c=Ae$. Linear absolute stability tests the amplification of a linear decay mode; nonlinear B-stability compares distances between two solutions of a dissipative vector field. Both describe propagation of perturbations, but they impose different conditions.

For the Dahlquist test equation, elimination of stages givesThe linear stability domain consists of $z$ for which the step is defined and $|R(z)|\le 1$. Poles or singular stage systems must be excluded, even if an output formula has a formal cancellation. For a scalar test equation, the proof is $y_n=R(z{)}^n{y}_0$: the powers are bounded exactly when $|R(z)|\le 1$. For $y^{\prime }=Ly$, the update is $R(hL)$. If $L$ is a normal matrix, unitary diagonalization of a normal matrix makes the operator norm equal to the largest scalar amplification modulus. For a diagonalizable matrix, powers are bounded by the condition number of the eigenvector matrix times the maximal scalar power. These constants must remain uniform for mesh-dependent systems. A defective unit-modulus eigenvalue can instead produce a growing Jordan block; the scalar spectral test alone does not control a general matrix.

An A-stable method accepts the entire closed left half-plane. The Explicit Euler method has $R(z)=1+z$ and a disk of stability $|1+z|\le 1$, so it is not A-stable. No nonconstant polynomial stability function can be A-stable, since it is unbounded on the negative real axis; this excludes all consistent explicit Runge-Kutta methods. The Backward Euler method has $R(z)=1/(1-z)$, and $|1-z|\ge 1$ when $\mathrm{Re}z\le 0$, proving A-stability. It is also L-stable, because $R(z)\to 0$ for large left-half-plane $z$.

The implicit midpoint rule and the trapezoidal rule have the same scalar stability function,The identity $|1-z/2{|}^2-|1+z/2{|}^2=-2\mathrm{Re}z$ proves A-stability. On the imaginary axis their amplification has unit modulus, and along the negative real axis it tends to $-1$. Thus neither is L-stable; very stiff decay can persist as alternating numerical values. In contrast, the two-stage Radau IIA method has $R(z)=(1+z/3)/(1-2z/3+z^2/6)$, with the explicit nonnegative modulus-gap proof in question 1, and is L-stable. This illustrates why L-stability matters beyond A-stability for a stiff differential equation.

For nonlinear problems, assume a real or complex Euclidean inner product and the dissipativity hypothesisfor all relevant states at each common time. Exact solutions satisfyA B-stable Runge-Kutta method reproduces this nonexpansion of distances whenever its stages exist. Put $d=y_n-{\tilde{y}}_n$, $D_i=Y_i-{\tilde{Y}}_i$, and $G_i=f(t_n+c_{i}h,Y_i)-f(t_n+c_{i}h,{\tilde{Y}}_i)$. Then $D_i=d+h{\sum }_j{a}_{ij}{G}_j$. Expansion of the output squared norm, followed by substituting $d=D_i-h{\sum }_j{a}_{ij}{G}_j$ in its linear terms, proves the Runge-Kutta contractivity identityIf $b_i\ge 0$ and $M=(m_{ij})$ is a positive semidefinite matrix, the first sum is nonpositive by dissipativity, and the second is nonnegative: expand each coordinate of the stage vectors to express it as a sum of nonnegative quadratic forms. These are exactly the conditions for algebraic stability of a Runge-Kutta method. Hence algebraic stability implies B-stability. Applying B-stability to the linear dissipative problem $y^{\prime }=\lambda y$, or its two-dimensional real form, also proves A-stability whenever stages are well defined.

The Backward Euler method has $b=A=1$, hence $M=1$; the implicit midpoint rule has $b=1$, $A=1/2$, hence $M=0$. Both are algebraically stable and B-stable. For the two-stage Radau IIA method,a positive semidefinite matrix, while both weights are positive. It therefore combines third-order accuracy, B-stability, and L-stability. The distinction between stability and solvability is essential: for Lipschitz continuous $f$, the stage map is a contraction when $hL{\max }_i{\sum }_j|a_{ij}|<1$, so the contraction mapping theorem proves small-step unique solvability. The preceding algebraic stability proof is an estimate for existing stages; it is not a theorem asserting stage existence for arbitrary step sizes and domains.

Linear A-stability is insufficient for nonlinear B-stability. The trapezoidal rule fails B-stability, even though its scalar stability function is identical to that of the B-stable implicit midpoint rule. Its Runge-Kutta method coefficients areFailure of this sufficient algebraic stability test alone would not prove failure of B-stability. An actual counterexample does: take $f(y)=-y^3$ and $h=2$. Dissipativity follows fromThe unique step map $T$ obeys $T+T^3=y-y^3$, because the left-hand side is strictly increasing and onto. At $y=1$, $T=0$, and implicit differentiation givesNearby initial states expand in distance. This proves a true nonlinear failure rather than merely failure of a coefficient criterion.

Finally, stability of a numerical method on finite intervals connects these contractivity properties to accuracy. If a one-step map is nonexpansive and its exact one-step defect has norm at most $C{h}^{p+1}$, then the error satisfies $\Vert e_{n+1}\Vert \le \Vert e_n\Vert +C{h}^{p+1}$, hence $\Vert e_n\Vert \le \Vert e_0\Vert +CT{h}^p$ for $nh\le T$. A Lipschitz step bound $1+Kh$ gives the same order using the discrete Gronwall inequality. For a stiff differential equation, the value of B-stability is that the propagation estimate need not grow with a large negative dissipative rate. The accuracy constant still needs appropriate smoothness and uniform derivative bounds; stability alone does not establish a mesh-uniform error order.

The usual sufficient input promises for the HHL algorithm are efficient quantum state preparation of the nonzero vector $b$, efficient access to a sparse matrix $A$ with at most $\mathrm{poly}(n)$ nonzero entries per row, and a condition number $\kappa ={\lambda }_{\max }/{\lambda }_{\min }=\mathrm{poly}(n)$ after an efficiently known normalization. Both the positions and values of the nonzero matrix elements must be computable coherently in polynomial time. More generally, efficient Hamiltonian simulation can replace the sparsity promise. We assume $A$ is invertible; if zero eigenvalues occur, one must instead restrict $b$ to their orthogonal complement and specify the desired inverse on that support.

With $a={\lambda }_{\max }$ and a known bound $\kappa$, choose $c=a/\kappa \le {\lambda }_{\min }$. The HHL controlled reciprocal rotation then succeeds with probabilityThis is the required inverse-polynomial lower bound. In the usual approximate algorithm, inverse-polynomial requested error also gives polynomial runtime under these input promises. The original HHL paper states the dependence on sparsity, conditioning, and accuracy.

There is a qualification for a literally exact version. Representability of the eigenvalues in $n$ bits does not itself make exact quantum phase estimation efficient: using $e^{2\pi iA}$ requires controlled powers corresponding to evolution times as large as $2\pi 2^{n-1}$. Under the paper's idealization we may describe their exact action, but polynomial runtime for that exact circuit additionally requires efficient implementations of those controlled powers, or equivalent efficient exact spectral access. Assuming an operation executes exactly does not bound its cost. This distinction is recorded in cost of exact phase estimation on a dyadic spectrum.

Starting from $x_0\in {\mathbb{R}}^n$, gradient descent repeatedly computes the gradient and updatesstopping when the gradient norm, step, or objective decrease is sufficiently small.

The Hessian bounds say that $f$ is $\alpha$-strongly convex and has $\beta$-smooth gradient. With $\eta =1/\beta$,Thus the iteration count isconvergence becomes slower linearly with the condition number $\kappa =\beta /\alpha$.

Forthe Hessian matrix is $\mathrm{diag}(2,200,20000)$, soTakeThenwhose Hessian is $2I$ and whose condition number is $1$.

The HHL algorithm requires coherent, efficient and repeatable preparation of the normalized state $|b⟩$, normally through a known preparation circuit and its inverse; possession of a single unknown physical specimen does not supply that access. The component of $b$ on any discarded or unresolved small-eigenvalue subspace must also be negligible. Here $U$ is a unitary operator, so it is invertible and all its singular values equal one, giving condition number $\kappa =1$.

Standard HHL is stated for a Hermitian matrix with an efficient sparse-access or block encoding oracle. A non-Hermitian $U$ can be embedded in the Hermitian block matrixpart (a) supplies efficient access to $U^{†}$. With inverse-polynomial target precision, $m=O(\log n)$ phase bits, and an efficient preparation oracle for $|b⟩$, the runtime is $\mathrm{poly}(n)$. The output is the normalized quantum state proportional to the solution $x$, rather than a classical list of all its amplitudes.

For the p-energythe first variation in the direction $\psi \in C_c^{\infty }(\Omega )$ isAn integration by parts therefore gives the Euler-Lagrange equationwhich is the p-Laplacian equation. In the notation of the question one takes $a^{ij}(x,z,\eta )=|\eta {|}^{p-2}{\delta }^{ij}$.

For $\eta \ne 0$, the principal coefficient matrix isIts eigenvalue in directions orthogonal to $\eta$ is $|\eta {|}^{p-2}$, while its eigenvalue parallel to $\eta$ is $(p-1)|\eta {|}^{p-2}$. The coefficients are $C^{1,\alpha }$ away from $\eta =0$, and the condition number there is at most $p-1$. On every region where $0<m\le |Du|\le M$, this gives uniform ellipticity with constants depending on $m$, $M$, and $p$. At $Du=0$ all principal eigenvalues vanish, so the operator is degenerate there and is not strictly elliptic on a domain containing a critical point.

For the HHL algorithm to have runtime polynomial in $\log N$, the Hermitian matrix $A$ must be invertible, have a condition number $\kappa$ bounded by $\mathrm{poly}(\log N)$, and be a sparse matrix with its nonzero entries efficiently accessible by an oracle. The normalized state $|b⟩$ must also be preparable in $\mathrm{poly}(\log N)$ time. With precision costs suppressed, these assumptions let HHL prepare, with high probability,

We fix the convention $PA=LU$: the permutation matrix $P$ acts on the rows of $A$, and the same row ordering must be applied to $b$ before the Triangular solves for LU decomposition. Equivalently, $A=P^{T}LU$, since $P^{-1}=P^T$. LAPACK documents its factors as $A=PLU$ instead, so check the defining equation rather than just the letter $P$.^[ref] Partial pivoting selects the largest absolute candidate in the current column, among rows $k,\dots ,n$, before forming elimination multipliers.^[ref]

A zero first pivot element does not imply a singular matrix. For example,Unpivoted Gaussian elimination would divide $2$ by zero. Indeed, no $A_0=LU$ with a unit lower triangular matrix $L$ exists: $u_{11}=0$ would force $(LU{)}_{21}=l_{21}{u}_{11}=0$, contradicting $(A_0{)}_{21}=2$. One row interchange resolves the obstruction. For $b_0=(1,5{)}^T$, use $P{b}_0=(5,1{)}^T$; back substitution then gives $x=(1,1{)}^T$. Using the original $b_0$ with these factors would solve a different linear system.

Partial pivoting bounds each elimination multiplier by one in absolute value in exact arithmetic. This helps control roundoff error, but intermediate entries can still grow; it is not an unconditional guarantee of numerical stability for every matrix.^[ref] It also does not remove the original problem's condition number. For the diagonal matrix $D=\mathrm{diag}(1,\epsilon )$ with $0<\epsilon <1$, the condition number in the Euclidean norm is ${\kappa }_2(D)=1/\epsilon$. Row permutations preserve this condition number, because they preserve Euclidean norms. Even an exact solve changes from $(1,1{)}^T$ to $(1,1+\eta /\epsilon {)}^T$ when the right-hand side changes from $(1,\epsilon {)}^T$ to $(1,\epsilon +\eta {)}^T$. This sensitivity is a property of the linear system, not a zero-pivot element failure.^[ref]

Suppose $A\in {\mathbb{C}}^{n\times n}$ is a diagonalizable matrix, $n\ge 2$, with an eigenbasis $v_1,\dots ,v_n$ and spectral decompositionAssume the dominant eigenvalue ${\lambda }_1$ is a simple eigenvalue andThen the normalized power iteration approaches the one-dimensional eigenspace of ${\lambda }_1$. Its asymptotic direction-error bound has factor $\gamma =|{\lambda }_2/{\lambda }_1|<1$. This spectral gap is a gap in modulus: if $g=|{\lambda }_1|-|{\lambda }_2|$, then $\gamma =1-g/|{\lambda }_1|$. A small relative gap gives slow convergence.^[ref]

The short proof is the exact eigenbasis identityThe remaining sum has norm bounded by a constant times ${\gamma }^k$. This proves convergence of direction, allowing a changing sign or complex factor. A tiny $c_1$ or a poorly conditioned, nonorthogonal eigenbasis can delay visible convergence: the condition number of $V$ affects constants, while $\gamma$ controls the eventual rate. The bound need not be an equality; if the initial coefficient of the second-largest eigenvalue vanishes, the active spectrum of the power method gives a faster rate. When $\gamma =0$, all unwanted components disappear after one multiplication.^[ref]

The hypothesis is $c_1=(V^{-1}{x}_0{)}_1\ne 0$, not an ordinary nonzero inner product with the right eigenvector. Let $w_1^{\ast }$ be the first row of $V^{-1}$, a left eigenvector normalized so $w_1^{\ast }{v}_j$ is $1$ for $j=1$ and $0$ otherwise; $\ast$ denotes conjugate transpose. Then $c_1=w_1^{\ast }{x}_0$. For a Hermitian matrix, choose an orthonormal eigenbasis and the criterion becomes $v_1^{\ast }{x}_0\ne 0$: the starting vector must be nonorthogonal to the dominant eigenvector. For a general diagonalizable matrix, the nonorthogonal eigenbasis makes these two tests different, as illustrated under power iteration failure cases.

Successive Givens rotations eliminate entries below the diagonal of a matrix. For an $m\times n$ matrix with $m\ge n$, process columns $k=1,\dots ,n$; within column $k$, rotate rows $k,j$ for each $j>k$ whose entry needs elimination. Apply each Givens row-pair update to columns $k,\dots ,n$. Both rows are zero in earlier columns, so these operations preserve earlier eliminations. No assumption of full column rank is needed to perform the eliminations.

For a concrete example, takeFirst use $G_1=G_{12}(3/5,4/5)$, then $G_2=G_{23}(3/\sqrt{13},2/\sqrt{13})$:The entry in position $(3,1)$ was already zero, so its elimination was skipped. The second Givens rotation combines the remaining pair $(3,2{)}^T$ in column two. This yields the full QR decompositionThe first two columns of $Q$ and first two rows of $R$ give the reduced QR decomposition. Direct multiplication verifies both $QR=A$ and $Q^{T}Q=I$. For these displayed factors evaluated in binary64 floating-point arithmetic, the Frobenius norms $\Vert A-QR{\Vert }_F$ and $\Vert Q^{T}Q-I{\Vert }_F$ are both below $10^{-14}$; these measure reconstruction and loss of orthogonality separately.

More generally, if the chronological rotations are $G_1,\dots ,G_p$, thenEach factor is an orthogonal matrix, so $Q$ is too. For a rectangular matrix, $R$ has zeros below its diagonal; its leading $n\times n$ block is an upper triangular matrix. With full column rank, retaining the first $n$ columns of $Q$ gives orthonormal columns. To impose a positive diagonal when necessary, flip matching columns of $Q$ and rows of $R$, preserving their product.

For a linear least-squares problem, apply the same chronological Givens rotations to the right-hand side $b$, obtaining $z=\Omega b$, without explicitly constructing $Q$. Since the Euclidean norm is preserved, solve the leading upper triangular matrix system $R_{1}x=z_{1:n}$ by back substitution when $A$ has full column rank; the remaining entries of $z$ give the minimum residual norm. In the example, $b=(0,0,1{)}^T$ gives $z=(0,2/\sqrt{13},3/\sqrt{13}{)}^T$, $x=(-8/65,2/13{)}^T$, and minimum residual Euclidean norm $3/\sqrt{13}$.

The locality is useful for a sparse matrix or an existing QR decomposition needing an extra row: only two rows change at each step, and the new row can be eliminated against the stored upper triangular matrix. For an $n\times n$ factor, this extra-row procedure takes $n$ rotations and $O(n^2)$ arithmetic operations. Locality does not guarantee preservation of sparsity: mixing rows can create nonzeros wherever either row had one. For a fully populated $m\times n$ input, the straightforward elimination uses ${\sum }_{k=1}^n(m-k)$ rotations, each touching $n-k+1$ columns, giving $O(m{n}^2)$ work. Storage layout, zero patterns and implementation affect speed; this count alone establishes no universal advantage over other QR decomposition algorithms.

The Euclidean norm preservation explains the appeal for numerical stability: exact rotations do not amplify a perturbation's length. Rounded coefficients and Givens row-pair updates still introduce roundoff error. Small reconstruction and orthogonality residuals check the computed factors, while the condition number of the original problem still controls how accurately a linear least-squares problem determines its solution.

For an invertible matrix $A$, the rank-one matrix update $B=A+u{v}^T$ is a singular matrix exactly when $1+v^T{A}^{-1}u=0$. This follows directly from the matrix determinant lemma. At that value, $u\ne 0$ and the nonzero vector $w=A^{-1}u$ satisfiesEvery vector in the kernel is a multiple of $w$, since $Bx=0$ implies $x=-A^{-1}u(v^{T}x)$. The matrix rank therefore drops by exactly one.

For $B(t)=I+tu{v}^T$ with $u=(1,2{)}^T$ and $v=(3,-1{)}^T$, the update becomes a singular matrix at $t=-1$, and $B(t)u=(1+t)u$. At $t=-0.99$, this direction is compressed by a factor $0.01$ and the inverse amplifies it by a factor 100. The actual spectral norm of the matrix inverse, which maximizes amplification over every direction, is about $700.17853409$.

The exact formula can expose a large condition number in the updated linear system even when the original one had a small condition number. It does not guarantee numerical stability merely because its evaluation uses fewer arithmetic operations. The denominator condition certifies existence; the condition number and floating-point behavior address different questions about using the resulting answer.

The linear-system residual $r^{(k)}=b-A{x}^{(k)}$ is computable without knowing $x_{\ast }$. The forward error $e^{(k)}=x^{(k)}-x_{\ast }$ is the actual difference from the desired solution. For an invertible matrix,Keeping $A$ fixed, the approximate vector exactly solves $A{x}^{(k)}=b+\delta b$ with $\delta b=-r^{(k)}$. Thus, for $b\ne 0$, $\Vert r^{(k)}\Vert /\Vert b\Vert$ is the relative backward error when only the right-hand side may change. For a vector norm and its induced operator norm,A small relative linear-system residual ensures a small relative forward error when the condition number is moderate; an ill-conditioned matrix can amplify it. The bound follows directly from $e=-A^{-1}r$ and $\Vert b\Vert =\Vert A{x}_{\ast }\Vert \le \Vert A\Vert \Vert x_{\ast }\Vert$.^[ref]

For the Jacobi iteration for a three-variable tridiagonal system, the first linear-system residual is $(1,3,1{)}^T$, while the first forward error is $(-1/2,-1,-1/2{)}^T$. In the Euclidean norm, the relative forward error is $\sqrt{3/28}\approx 0.3273$ and the relative linear-system residual is $\sqrt{11/120}\approx 0.3028$. Its spectral condition number of a positive-definite matrix is $(4+\sqrt{2})/(4-\sqrt{2})\approx 2.094$, so the bound is about $0.6340$. The bound is a guarantee, not an equality. In floating-point arithmetic, recompute the linear-system residual from the current vector when checking termination; a small update alone need not certify a small forward error.

Right preconditioning substitutes $x=M^{-1}u$ into the linear system $Ax=b$. Applying its transformed linear map first solves $Mz=v$, then computes $Az$. Recover the solution by solving $Mx=u$. Its linear-system residual $b-A{M}^{-1}u$ is exactly the original linear-system residual $b-Ax$, which is useful when monitoring the imbalance in the original equations.

The left preconditioning and right preconditioning transformed matrices are similar matrices:They therefore have the same eigenvalues, although their Euclidean norm condition numbers and residual histories need not agree. A similarity transformation preserves eigenvalues; a nonorthogonal change of coordinates need not preserve lengths.

Round-off error arises when numbers or arithmetic results are rounded to a finite set of representable values. Its scale is associated with machine epsilon, and its effects depend on subsequent operations. Even a formula with zero truncation error can lose information. In decimal arithmetic rounded to three significant digits, $1.234$ becomes $1.23$ and $1.233$ becomes $1.23$: subtracting the stored numbers gives zero although the exact difference is $0.001$. The subtraction exposes errors already present in its nearly equal operands.

This example also has a large condition number: independently perturbing the two inputs of $a-b$ can produce a relative amplification $(|a|+|b|)/|a-b|=2467$. High sensitivity belongs to the input-output problem; numerical stability concerns the arithmetic used to solve it. Conversely, on a common binary system with 53 significant bits and rounding to nearest, $(10^{16}+1)-10^{16}$ evaluates to zero while $(10^{16}-10^{16})+1$ evaluates to one. Exact arithmetic makes both one: changing summation order changes the accumulated roundoff error. MIT's summation lecture explains why numerical stability and the condition number must be assessed separately.^[ref]

A matrix over a field $\mathbb{F}$ is in row echelon form when all zero rows follow the nonzero rows, each nonzero row's first nonzero entry lies strictly to the right of the preceding row's first nonzero entry, and all entries below each such leading entry vanish. These leading entries are its pivot elements. They need not be one, and entries above them need not vanish.^[ref]

The staircase turns a linear system into a sequence of simpler equations. For example,is in row echelon form, with pivot columns 1 and 2. Its second equation determines $x_2$ once $x_3$ is chosen; its first then determines $x_1$ by back substitution. It is not in reduced row echelon form, because the second pivot column contains a nonzero entry above its pivot element. Neither being square nor having a nonzero entry in every row is required.

Elementary row operations bring every matrix to row echelon form: choose a nonzero entry in the leftmost available nonzero column, interchange rows, clear entries below it, and repeat on the remaining rows and later columns. Skip columns with no remaining nonzero entry. Clearing above each pivot element and scaling it to one completes reduced row echelon form.^[ref]

The nonzero rows of an row echelon form are linearly independent: in a vanishing linear combination, inspecting the first pivot column forces the first coefficient to be zero, then the next forces the second, and so on. They give a basis of the original row space. The number of pivot elements is both the row rank and the column rank. To recover a basis of the original column space, select the corresponding original pivot columns; the reduced column vectors generally belong to a different column space.

Here the arithmetic is exact in the specified field. For floating-point arithmetic, deciding whether a computed entry counts as zero requires a numerical criterion; see Tiny-pivot roundoff in Gaussian elimination and condition number. An exact row echelon form classification alone does not settle numerical stability.

The displayed Thomas algorithm requires every modified pivot element to be nonzero in exact arithmetic. Nonzero original diagonal entries alone are insufficient: later pivot elements can vanish after elimination. Two useful sufficient contexts are a real symmetric positive-definite matrix, and strict diagonal dominance by rows:For the symmetric positive-definite matrix case, elimination leaves positive-definite remaining blocks by the Schur complement, so each $p_i>0$. Equivalently, the initial principal minors ${\Delta }_i=\det A_{1:i,1:i}$ are positive, and $p_i={\Delta }_i/{\Delta }_{i-1}$ with ${\Delta }_0=1$.

For strict diagonal dominance, a short induction shows $|p_i|>|c_i|$. The first row gives $|p_1|=|b_1|>|c_1|$. Assuming the preceding bound, the triangle inequality givesIn particular $p_n\ne 0$. These are structural guarantees against a zero pivot element, not universal bounds on forward error in floating-point arithmetic. A poorly conditioned linear system can still amplify roundoff error; numerical stability and the condition number must be distinguished.

An $n\times n$ square matrix $A$ over a field, with $n\ge 1$, is singular when it has no matrix inverse. Geometrically, its linear map loses at least one input direction: some nonzero vector $v$ satisfies $Av=0$, so $A(x+v)=Ax$. Over the real numbers, the following family makes that loss visible:

For this diagonal matrix, $\det A(t)=t$. Whenever $t\ne 0$, the input can be recovered as $(x,y)=(b_1,b_2/t)$ and the kernel is $\{0\}$. At $t=0$, the kernel becomes $\mathrm{span}\{(0,1)\}$: the entire second coordinate is forgotten. Consequently $A(0)x=(b_1,b_2)$ has no solution if $b_2\ne 0$, and has the solution straight line $x=(b_1,s)$, $s\in \mathbb{R}$, if $b_2=0$. A singular matrix therefore need not make a particular linear system inconsistent; it prevents unique recovery when that linear system is consistent.

For an $n\times n$ square matrix, all these tests describe the same failure:Here $\mathbb{F}$ is the field of scalars. Equivalently, the columns are linearly dependent, the linear map is neither injective nor surjective, and some outputs cannot be reached. See the invertible matrix equivalences and Strang's MIT 18.06SC lecture summary, Multiplication and inverse matrices, p. 2. Over an infinite field, every consistent linear system for a singular matrix has infinitely many solutions; over a finite field with $q$ elements and nullity $d$, it has exactly $q^d$ solutions. This follows by adding each kernel vector to one solution.

The square hypothesis is essential. For an $m\times n$ matrix, injectivity means matrix rank $n$ (full column rank), whereas surjectivity means matrix rank $m$ (full row rank). A rectangular matrix can satisfy one condition without the other; its kernel and image of a linear map give the useful tests without calling it a singular matrix.

Exact singularity also differs from sensitivity. For $0<|t|\le 1$, $A(t)$ is an invertible matrix, but its smallest singular value is $|t|$ and its condition number in Euclidean norm is $1/|t|$. A change $\delta b=(0,\epsilon )$ changes the recovered input by $\delta x=(0,\epsilon /t)$: with $t=1/100$ and $\epsilon =1/1000$, an output change of $0.001$ causes an input change of $0.1$. These are exact toy values, not measurements. The condition number quantifies this sensitivity before a direction is lost; see Steve Marschner, Condition numbers, §2. A small determinant alone is not a reliable sensitivity test: $\epsilon I_2$ has determinant ${\epsilon }^2$ yet condition number one for every nonzero real $\epsilon$, since both singular values shrink equally.

The singular values of a matrix measure stretching of Euclidean norm. They are the nonnegative square roots of the eigenvalues of $A^{\ast }A$, counting the $\min (m,n)$ diagonal slots of the singular value decomposition; when $n>m$, $A^{\ast }A$ has additional zero eigenvalues. Nonnegativity follows from $x^{\ast }{A}^{\ast }Ax=\Vert Ax{\Vert }_2^2\ge 0$, so $A^{\ast }A$ is a positive semidefinite matrix. For the preceding $2\times 2$ matrix,Both have eigenvalues $25,1$, giving singular values $5,1$. In particular,Here the operator norm uses Euclidean norm: no unit vector stretches more than fivefold. Zero singular values identify lost directions; the number of positive singular values equals the matrix rank. This summarizes both amplification and information loss with nonnegative numbers.^[ref]

Small positive singular values also explain sensitivity in an inverse problem. For a square invertible matrix, the matrix inverse sends $u_j$ to $v_j/{\sigma }_j$: a weak forward response becomes strong inverse amplification. The condition number for Euclidean norm is ${\sigma }_1/{\sigma }_n$, equal to $5$ in our example. To see a larger effect, keep the same right singular vectors but change the second singular value to $\epsilon =0.01$:The output perturbation has Euclidean norm $0.001$, but the recovered input changes by $0.1$. Its hundredfold amplification follows from the direction of the perturbation, not just its size. The condition number is now $500$; it measures the disparity between the strongest and weakest responses.^[ref]

Let $A$ and $M$ be real symmetric positive-definite matrices. The principal square root of a positive semidefinite matrix construction gives an invertible matrix for each. SetThen $Ax=b$ is equivalent to $By=c$. This is a matrix congruence, and $B$ is a symmetric positive-definite matrix: $B^T=B$, and for every nonzero vector $v$,since $S^{-1}v\ne 0$. This short proof establishes both requirements for applying the Conjugate gradient method to the transformed linear system.

The left transformed matrix has the same eigenvalues becauseIts correct geometry uses the weighted inner product $⟨u,v{⟩}_M=u^{T}Mv$: $⟨u,M^{-1}Av{⟩}_M=u^{T}Av=⟨M^{-1}Au,v{⟩}_M$. Thus it is symmetric in this inner product, even when it is not a symmetric matrix in ordinary coordinates. The spectral condition number governing the transformed symmetric problem is ${\kappa }_2(B)$, not generally ${\kappa }_2(M^{-1}A)$.

The Preconditioned conjugate gradient method realizes this transformation using auxiliary solves $M{z}_k=r_k$, without constructing $S$ or its matrix inverse. With fixed $M$, initialize $r_0=b-A{x}_0$ and $p_0=z_0$; while $r_k\ne 0$, usethen solve $M{z}_{k+1}=r_{k+1}$ and setStop at a zero linear-system residual before forming another ratio. Positive definiteness makes the denominators positive before termination.^[ref]

An invertible matrix alone is insufficient: with $A=I$ and $M=\mathrm{diag}(-1,1)$, the left transformed matrix has eigenvalues $-1,1$, and $M$ has no real symmetric positive-definite matrix square root. Even a valid symmetric positive-definite matrix can worsen conditioning if its scaling is unsuitable, as the next worked example shows.

A nonzero pivot element can still be a poor numerical choice. Partial pivoting bounds the absolute values of the elimination multipliers by one in exact arithmetic, because each numerator is no larger than the selected denominator. This usually improves numerical stability, but does not prevent every possible growth of intermediate entries or guarantee accurate answers for every linear system.^[ref]

To see the mechanism without depending on hardware, define a toy floating-point arithmetic model. The operator ${\mathrm{fl}}_3(z)$ rounds an exact real number to the nearest value with three significant decimal digits; halfway cases choose the even final digit, and ${\mathrm{fl}}_3(0)=0$. Allow an unbounded exponent range. Round the inputs and the result of every addition, subtraction, multiplication and division separately; a product followed by a subtraction rounds twice. This is a prescribed arithmetic model, not a measurement of a physical computer. All inputs in the following original example are represented exactly:Without a row interchange, the elimination multiplier is $m={\mathrm{fl}}_3(1/\epsilon )=10^6$. The last coefficient and right-hand side becomeTwo distinct exact values now have the same rounded representation. Back substitution therefore returnsThe small difference carrying information about $x_1$ was lost during elimination; the final subtraction only exposes that loss. Dividing by the tiny pivot element amplifies sensitivity to the lost information.

With partial pivoting, interchange the two rows, including their right-hand sides, to start from $\left[\begin{array}{ccc}1& 1& 2\\ \epsilon & 1& 1\end{array}\right]$. Now $m={\mathrm{fl}}_3(\epsilon /1)=\epsilon$, and the rounded updates areBack substitution gives ${\hat{x}}_2={\mathrm{fl}}_3(1/1)=1$ and ${\hat{x}}_1={\mathrm{fl}}_3({\mathrm{fl}}_3(2-{\mathrm{fl}}_3(1\cdot 1))/1)=1$. Both paths round intermediate values, but only the first creates million-sized entries and then divides a lost difference by $10^{-6}$.

The true solution follows from the original second equation $x_2=2-x_1$:Evaluate the linear-system residual $r=b-A\hat{x}$ against the original data in exact arithmetic, separately from the three-digit solve. With $\Vert v{\Vert }_{\infty }={\max }_i|v_i|$, the maximum norm, the results areThe respective relative linear-system residuals $\Vert r{\Vert }_{\infty }/\Vert b{\Vert }_{\infty }$ are $1/2$ and $5\times 10^{-7}$. Computing the diagnostic in higher precision matters: a check made only against the rounded reduced equations would miss the original error.

This matrix is well conditioned in the maximum norm: its induced matrix norm is $\Vert A{\Vert }_{\infty }=2$, whileThe failure therefore belongs to this unpivoted algorithm, rather than a large condition number. In general, however, partial pivoting cannot remove sensitivity inherent in the data. For any approximate solution of an invertible matrix system, $\hat{x}-x_{\ast }=-A^{-1}r$; the induced matrix norm inequality and $\Vert b\Vert \le \Vert A\Vert \Vert x_{\ast }\Vert$ giveA small linear-system residual controls solution error only in conjunction with the condition number; numerical stability and sensitivity are different questions.^[ref]

A linear map $T:V\to W$ has a trivial kernel when $\ker T=\{0_V\}$: the zero vector is the only input producing the zero output. The kernel is a singleton set, not the empty set, because every linear map sends zero to zero. Equivalently, its nullity is zero and $T$ is injective. This holds even without a finite-dimensional hypothesis; see Linear Algebra Done Right by Sheldon Axler (4th edition).

The reason this zero-output test controls all outputs is linearity:With a trivial kernel, equal outputs force equal inputs. Therefore every equation $T(x)=b$ has at most one solution. Existence is a separate question: $b$ must belong to the image of a linear map. In the fiber description under nullity, $x_0+\{0\}$ is just the single point $x_0$.

For example, the linear map $J:{\mathbb{R}}^2\to {\mathbb{R}}^3$, $J(x,y)=(x,y,0)$, has trivial kernel and full column rank:The output $(3,4,0)$ comes from exactly $(3,4)$, while $(3,4,1)$ is unreachable. Conversely, $P:{\mathbb{R}}^3\to {\mathbb{R}}^2$, $P(x,y,z)=(x,y)$, is surjective and has full row rank, but its kernel is $\mathrm{span}\{(0,0,1)\}$: every output $(a,b)$ comes from the solution straight line $(a,b,s)$.

For an $m\times n$ matrix, a trivial kernel is equivalent to matrix rank $n$ and linear independence of its columns. It can therefore occur only if $n\le m$. If $m=n$, the rank-nullity theorem also forces surjectivity, giving an invertible matrix; if $m>n$, the example $J$ shows why unique recovery of reachable inputs does not imply that every output is reachable. Finally, a trivial kernel guarantees exact uniqueness without guaranteeing numerical stability: the singular matrix example $A(t)$ has trivial kernel for every $t\ne 0$, even as its condition number grows without bound when $t$ approaches zero.