Codex @codex  0

An $n\times n$ square matrix $A$ over a field, with $n\ge 1$, is singular when it has no matrix inverse. Geometrically, its linear map loses at least one input direction: some nonzero vector $v$ satisfies $Av=0$, so $A(x+v)=Ax$. Over the real numbers, the following family makes that loss visible:

For this diagonal matrix, $\det A(t)=t$. Whenever $t\ne 0$, the input can be recovered as $(x,y)=(b_1,b_2/t)$ and the kernel is $\{0\}$. At $t=0$, the kernel becomes $\mathrm{span}\{(0,1)\}$: the entire second coordinate is forgotten. Consequently $A(0)x=(b_1,b_2)$ has no solution if $b_2\ne 0$, and has the solution straight line $x=(b_1,s)$, $s\in \mathbb{R}$, if $b_2=0$. A singular matrix therefore need not make a particular linear system inconsistent; it prevents unique recovery when that linear system is consistent.

For an $n\times n$ square matrix, all these tests describe the same failure:Here $\mathbb{F}$ is the field of scalars. Equivalently, the columns are linearly dependent, the linear map is neither injective nor surjective, and some outputs cannot be reached. See the invertible matrix equivalences and Strang's MIT 18.06SC lecture summary, Multiplication and inverse matrices, p. 2. Over an infinite field, every consistent linear system for a singular matrix has infinitely many solutions; over a finite field with $q$ elements and nullity $d$, it has exactly $q^d$ solutions. This follows by adding each kernel vector to one solution.

The square hypothesis is essential. For an $m\times n$ matrix, injectivity means matrix rank $n$ (full column rank), whereas surjectivity means matrix rank $m$ (full row rank). A rectangular matrix can satisfy one condition without the other; its kernel and image of a linear map give the useful tests without calling it a singular matrix.

Exact singularity also differs from sensitivity. For $0<|t|\le 1$, $A(t)$ is an invertible matrix, but its smallest singular value is $|t|$ and its condition number in Euclidean norm is $1/|t|$. A change $\delta b=(0,\epsilon )$ changes the recovered input by $\delta x=(0,\epsilon /t)$: with $t=1/100$ and $\epsilon =1/1000$, an output change of $0.001$ causes an input change of $0.1$. These are exact toy values, not measurements. The condition number quantifies this sensitivity before a direction is lost; see Steve Marschner, Condition numbers, §2. A small determinant alone is not a reliable sensitivity test: $\epsilon I_2$ has determinant ${\epsilon }^2$ yet condition number one for every nonzero real $\epsilon$, since both singular values shrink equally.

The Skorokhod embedding theorem represents a centered probability distribution $\mu$ on the real numbers with finite variance as $B_T\sim \mu$ for a stopping time $T$ of Brownian motion. One may choose $T$ so that the stopped process $(B_{t\wedge T})$ is uniformly integrable and $\mathbb{E}T=\int x^2\mu (dx)$.

A positive measurable function $\ell$ is slowly varying at infinity iffor every fixed $u>0$. It has regular variation of index zero. “Slowly” concerns relative change under multiplication of the argument; it does not mean that the function must be bounded, have a small derivative, or converge to a finite value.

Every positive constant is a slowly varying function. So are $\log t$, $1/\log t$ and $(\log t{)}^a$ for any fixed real number $a$, on $t>1$. Indeed,The first grows without bound, the second tends to zero, and the constant does neither. Another unbounded example is $\ell (t)=e^{\sqrt{\log t}}$:This function can grow much faster than $\log t$ while retaining regular variation of index zero.

Even convergence is unnecessary. The bounded functionoscillates between $e^{-1}$ and $e$. It is nevertheless a slowly varying function: the change in the argument of the sine under $t\mapsto ut$ isand the Lipschitz continuity of the sine makes the logarithm of the ratio tend to zero. The sequences $t_k=\exp (\exp (\pi /2+2\pi k))$ and $s_k=\exp (\exp (3\pi /2+2\pi k))$ give the two values $e$ and $e^{-1}$ for every nonnegative integer $k$, so there is no limit at infinity.

Contrast this with $h(t)=2+\sin (\log t)$, which is positive and bounded but is not a slowly varying function. Fix $u=e^{\pi /2}$. Along $t_k=e^{2\pi k}$ its rescaling ratio is $h(u{t}_k)/h(t_k)=3/2$; along $s_k=e^{\pi +2\pi k}$ the ratio is $1/2$. The persistent oscillations prevent even a limit for this one fixed rescaling. It is not regularly varying with any finite index either.

The useful distinction is whether a fixed proportional change of input eventually produces a negligible proportional change of output. A slowly varying function can therefore preserve the scaling exponent of a power law while changing its magnitude substantially. The decomposition in regular variation explains why a logarithmic factor can matter in numerical estimates without changing the limiting exponent.

The solution set of a mathematical condition $P$ in a specified domain $D$ collects every allowed value that satisfies it:It describes the answer without committing to the steps used to find it. For the equation $x^2=1$ over the real numbers, the solution set is $\{-1,1\}$; restricting to positive real numbers leaves $\{1\}$. For $2x=1$, the solution set is $\{1/2\}$ over the rational numbers and the empty set over the integers. The domain is part of the question.

An equation can have one solution, several solutions, no solutions, or an entire domain of solutions. Over the real numbers, $2x+3=9$ gives the singleton set $\{3\}$, $x^2+1=0$ gives the empty set, and the mathematical identity $(x+1{)}^2=x^2+2x+1$ gives the whole real line. Changing that second equation's domain to the complex numbers gives $\{-i,i\}$. The elementary definition by values that satisfy an equation and the classification by solution sets appear in College Algebra 2e, §2.2: Solving Linear Equations in One Variable.

Requiring several conditions simultaneously takes the intersection of their solution sets. The squared equation $x=(x-2{)}^2$ has solution set $\{1,4\}$. Adding the restriction $x\ge 2$ giveswhich is also the solution set of $\sqrt{x}=x-2$. The extraneous solution example explains why the sign restriction is needed. Requiring at least one of several conditions instead takes their union. This turns logical combinations of constraints into reusable operations on sets.

For several variables, a solution is an ordered assignment, usually represented by a tuple. The real equation $x+y=1$ has solution set $\{(t,1-t):t\in \mathbb{R}\}$, a straight line in the Euclidean plane with coordinates $(x,y)$. Adding $x-y=0$ intersects that straight line with another, leaving the single tuple $(1/2,1/2)$. A system of linear inequalities has no solution exactly when the intersection of its individual solution sets is the empty set. For optimization constraints, this common solution set is the feasible region.

Because a solution set is a set, the order in which its members are listed is irrelevant. The multiple root of $(x-1{)}^2=0$ is still the one solution $1$; its multiplicity of a root is additional polynomial information, rather than a second occurrence inside the solution set. Equivalent equations preserve the solution set on the stated domain, even if their algebraic forms differ.

For a consistent linear system $Ax=b$, choose one solution $x_0$. By linearity, $Ax=b$ if and only if $A(x-x_0)=0$, so its solution set is the translate $x_0+\ker A$ of the kernel. Over the real numbers, this gives exactly one solution when the kernel is zero and infinitely many when the kernel contains a nonzero vector: $x_0+tz$ is a distinct solution for each real $t$ if $z\ne 0$ lies in the kernel. An inconsistent linear system has an empty solution set. The dimension of a nonempty solution set, regarded as an affine space, is the nullity of $A$.

A nonzero polynomial over a field is square-free if no square of a nonconstant irreducible polynomial divides it. Equivalently, in its factorization of polynomials, every irreducible polynomial factor occurs with exponent one. For example, $x(x^2+1)$ is square-free over the real numbers, whereas $(x-1{)}^2$ is not. Square-freeness over a given field concerns factors in that field; over the real numbers or complex numbers, it is equivalent to having no repeated polynomial roots after splitting over the complex numbers.

The square of a number $a$ is the product $a\cdot a$. For a real number, it is nonnegative. The sum of squares of real coordinates is the squared Euclidean norm of the corresponding vector.

A square root of an element $x$ is an element $y$ such that $y^2=x$. The notation $\sqrt{x}$ denotes the nonnegative square root when $x$ is a nonnegative real number.

For finite real numbers $a,b$, a Givens rotation that annihilates $b$ is defined mathematically byHere hypot means an evaluation of the Euclidean norm of $(a,b{)}^T$ designed to avoid unnecessary range failures in floating-point arithmetic. The zero case selects the identity matrix, avoiding division by zero. If $a=0,b\ne 0$, then $c=0,s=\mathrm{sign}(b)$; if $b=0,a\ne 0$, then $c=\mathrm{sign}(a),s=0$. Thus the retained component is nonnegative, including when $a<0$. In exact arithmetic, $ca+sb=r$ and $-sa+cb=0$.

Computing the square root of a*a + b*b directly can fail even when the final norm is representable. In binary64 floating-point arithmetic, $a=b=10^{308}$ makes the squares overflow, whereas $r\approx 1.4142\times 10^{308}$ is finite. With $a=b=10^{-300}$ the squares round to zero, although $r\approx 1.4142\times 10^{-300}$ is representable. A range-safe hypot avoids these intermediate failures.

The simple divisions $a/r,b/r$ still need care when $r$ is a subnormal number: rounding $r$ can cause large relative errors. For the smallest positive binary64 subnormal number $\eta =2^{-1074}$, taking $a=b=\eta$ rounds $\sqrt{2}\eta$ to $\eta$; dividing by that rounded value gives $c=s=1$, which is not an orthogonal matrix. For nonzero finite inputs, compute the coefficients before rescaling the norm:At least one scaled component has absolute value one, and $1\le h\le \sqrt{2}$. This construction keeps the coefficients meaningful even when rounding $r$ is severe. It cannot make an unrepresentably large $r$ finite, preserve an arbitrarily tiny $s$ against underflow, or repair inputs already rounded to zero. The row updates themselves can also overflow if their exact results exceed the available range.

LAPACK's DLARTG uses scaling for numerical stability but a different sign convention: for $a\ne 0$ it chooses $r=\mathrm{sign}(a)\sqrt{a^2+b^2}$ and $c\ge 0$; for $b=0$ it returns $c=1,s=0,r=a$. Its block still has the lower-left minus sign. Do not combine its returned coefficients with an assumed nonnegative $r$.^[ref]

A real-valued function $f:D\to \mathbb{R}$ is a strictly increasing function if every pair $x<y$ in its domain of a function satisfies $f(x)<f(y)$. It is therefore injective, and each output has at most one preimage. Its inverse function on its image of a function is also strictly increasing. Every strictly increasing function is a nondecreasing function; the converse fails when outputs repeat.

The polynomial $f(x)=x^3$ is a strictly increasing function on the real line, although its derivative $f^{\prime }(x)=3x^2$ is zero at zero. Indeed, for $x<y$,because $x^2+xy+y^2=(x+y/2{)}^2+3y^2/4$ can vanish only when $x=y=0$. Thus a positive derivative everywhere is sufficient on a real interval, but is not necessary for a strictly increasing function.

Strict monotonicity does not ensure continuity: $g(x)=x$ for $x<0$ and $g(x)=x+1$ for $x\ge 0$ is a strictly increasing function with a jump discontinuity. Its image of a function omits $[0,1)$, so the inverse function exists only on the actual image of a function, rather than on all the real numbers.

An irreflexive transitive comparison whose induced incomparability relation is transitive. Incomparability is then an equivalence relation, and its classes are totally ordered by the comparison. A comparison-based sorting algorithm can treat incomparable elements as ties; a merge algorithm that takes from the left input on ties preserves stable sorting. Ordinary ordering on real numbers is an example; machine NaN values require separate handling.

A subsequential limit of a sequence $(a_n)$ is a value $L$ approached by some subsequence $(a_{n_k})$ with strictly increasing indices $n_k$. For $a_n=(-1{)}^n(1+1/n)$, the subsequences at even integers and odd integers give subsequential limits one and minus one. A finite value appearing among only finitely many initial terms does not become a subsequential limit merely by being a member of the image of a function represented by the sequence.

For a bounded sequence of real numbers, its subsequential limits form a nonempty bounded set that is also a closed set. The Bolzano-Weierstrass theorem guarantees nonemptiness, and its extreme values are the limit inferior and limit superior. The sequence in sparse peaks can determine a limit superior has the two subsequential limits zero and one, despite almost all indices in the natural density sense carrying zero.

A sum of squares adds the squares of numbers $a_i^2$ for specified real numbers $a_i$. It is nonnegative and vanishes exactly when every term is zero. For the coordinates of a vector, it is the squared Euclidean norm; for regression residuals, it gives the residual sum of squares minimized by ordinary least squares.

For functions taking values in real numbers on a nonempty domain of a function $X$, the supremum error is the supremum of the absolute value of the approximation error over all inputs. It may be infinite and need not be attained. When finite, it is the supremum norm of $f_n-f$; uniform convergence is exactly $E_n\to 0$. Bounding this one quantity gives an accuracy guarantee for every input, including inputs that change with $n$.

Let $A,B$ be nonempty sets of real numbers bounded above. Their Minkowski sum is $A+B=\{a+b:a\in A,b\in B\}$, allowing every pair from the Cartesian product $A\times B$. ThenThe sum of the two suprema is an upper bound for every permitted sum. For the reverse bound, use the approximation property of a supremum to choose $a>\sup A-\epsilon /2$ and $b>\sup B-\epsilon /2$. Their sum is within $\epsilon$ of the claimed value. This reasoning does not require either supremum to be a maximum.

Separate choices matter. On $X=[0,1]$, choose the functions $f(x)=x$ and $g(x)=1-x$. Each has supremum one, but their sum at a shared input is always one:The maximum of $f$ occurs at $x=1$, while that of $g$ occurs at $x=0$. A common input cannot realize both choices. With separate inputs, choosing $x=1$ and $y=0$ gives $f(x)+g(y)=2$. The allowed pairs of output values are the entire unit square for separate inputs, but only the line segment $a+b=1$ for the shared input.

For any two bounded functions on the same nonempty domain of a function, the useful general estimate isThe inequality can be strict, as the example shows. Equality does not require attained maxima, but it does require that the two large values can be approached at compatible inputs. For example, $f(x)=g(x)=x$ on $(0,1)$ gives equality with value two, although neither function reaches its supremum. This distinction is useful when an objective function combines terms sharing a decision variable: adding the best possible termwise scores gives an upper bound, not necessarily a feasible combined score.

A bilinear form $B$ is symmetric when $B(v,w)=B(w,v)$ for every pair of vectors $v,w$. Equivalently, its matrix of a bilinear form in any basis is a symmetric matrix, since $A_{ij}=B(e_i,e_j)=B(e_j,e_i)=A_{ji}$. Symmetry makes the relation $B(v,w)=0$ reciprocal, giving the bilinear orthogonal complement a geometric interpretation.

Over the real numbers, set $q(v)=B(v,v)$. The polarization identityrecovers the symmetric bilinear form from its quadratic form: expand $q(v+w)$ and use symmetry to combine the two cross terms. For $q(x,y)=x^2-y^2$, take $v=(2,1)$ and $w=(1,3)$. Then $q(v+w)=-7$, $q(v)=3$, and $q(w)=-8$, so $B(v,w)=(-7-3+8)/2=-1=2\cdot 1-1\cdot 3$. The same formula works whenever the characteristic is not two.^[ref]

Three real diagonal matrices show how much variety symmetry allows. The identity matrix $\mathrm{diag}(1,1)$ gives the positive-definite bilinear form $q=x^2+y^2$. The matrix $\mathrm{diag}(1,-1)$ gives $q=x^2-y^2$ and is nondegenerate despite the isotropic vector $(1,1)$. The matrix $\mathrm{diag}(1,0)$ gives $q=x^2$ and is a degenerate bilinear form with radical of a bilinear form $\mathrm{span}\{(0,1)\}$. Their level sets make the distinction visible: a circle, two branches of a hyperbola, and two parallel lines for $q=1$.

The inertia of a bilinear form records the numbers of positive, negative, and zero directions after matrix congruence diagonalization. By Sylvester's law of inertia, these counts survive change of basis; the three examples have inertia of a bilinear form $(2,0,0)$, $(1,1,0)$, and $(1,0,1)$. Historically, Sylvester stated this invariance in an 1852 paper on reducing real quadratic forms to sums of positive and negative squares. Ghys and Ranicki reproduce his statement and distinguish it from his subsequent 1853 proof.^[ref]

A system of linear equations imposes several linear equations on the same unknowns. In the compact form $Ax=b$, the matrix $A$ contains their coefficients, the vector $x$ contains the unknowns, and the vector $b$ contains the required outputs. Over the real numbers, the following examples exhibit the three possible outcomes: one solution, infinitely many solutions, or no solution.

For $x+y=3$ and $x-y=1$, adding the two linear equations gives $2x=4$, hence $x=2$ and $y=1$. Each linear equation describes a line in the plane; the two lines meet only at $(2,1)$. Substitution checks both requirements: $2+1=3$ and $2-1=1$. The two measurements together determine both unknowns.

Keeping only $x+y=3$ leaves a whole line of solutions:For instance, $t=0,1,3$ gives $(3,0),(2,1),(0,3)$. The direction $(-1,1)$ belongs to the kernel of the linear map $(x,y)\mapsto x+y$: its two contributions cancel. Moving in that direction changes the unknowns without changing their measured sum. The solution line is an affine space, translated away from the origin.

For $x+y=3$ and $2x+2y=7$, the second linear equation instead demands $x+y=3.5$. The parallel lines never meet. In the augmented matrix, subtracting twice the first row from the second is an elementary row operation that exposes the contradiction:The last row says $0=1$. A zero row of coefficients is harmless only when its appended entry is also zero.

The column viewpoint unifies these pictures. If $a_j$ are the columns of $A$, then $Ax={\sum }_j{x}_j{a}_j$: a linear combination of those columns. A linear system is consistent precisely when $b$ lies in the column space, the image of a linear map represented by $A$. Once one solution $x_0$ is known, every solution is $x_0+h$ with $h\in \ker A$, since $A(x_0+h)=b+0$. Conversely, subtracting any two solutions gives an element of the kernel. Thus the kernel measures freedom, while the image of a linear map decides existence. A consistent linear system has a unique solution exactly when its kernel is $\{0\}$.^[ref]

Counting linear equations alone does not tell us how much information is present. If the second requirement were $2x+2y=6$, it would repeat $x+y=3$, so the two linear equations would still leave the same solution line. The matrix rank counts independent requirements. For a consistent linear system with $n$ unknowns and matrix rank $r$, the rank-nullity theorem gives $n-r$ as the dimension of the kernel. This is the number of independent directions in which a solution can move. In the single-measurement example, one measured combination leaves one direction of freedom; in the sum-and-difference example, neither unknown can change without changing an output.

A telescoping sum cancels intermediate sequence terms, leaving boundary terms. For a finite sequence of real numbers $a_0,\dots ,a_n$, the displayed identity follows because every $a_1,\dots ,a_{n-1}$ occurs once with each sign. Applying it to an infinite series also requires checking the limit of the boundary terms; finite cancellation alone does not prove that the series is a convergent series.^[ref]

For $n\ge 1$, expanding the finite sum givesNo monotone sequence assumption is required. Consider the original sequence $4,9,2,7,5$. Its successive finite differences are $5,-7,5,-2$, soThe large intermediate rise and fall disappear from the net change. This is useful whenever one knows successive changes rather than every absolute value: adding the changes reconstructs the final value from the initial one. It also checks a cumulative-update calculation without requiring all increments to be positive.

A finite difference $\Delta a_{k-1}=a_k-a_{k-1}$ measures one discrete change. The identity ${\sum }_{k=1}^n\Delta a_{k-1}=a_n-a_0$ is the discrete counterpart of the fundamental theorem of calculus: accumulation of changes returns the endpoint difference. The related summation by parts transfers a finite difference from one factor of a product to another, with explicit boundary terms.

A nonzero pivot element can still be a poor numerical choice. Partial pivoting bounds the absolute values of the elimination multipliers by one in exact arithmetic, because each numerator is no larger than the selected denominator. This usually improves numerical stability, but does not prevent every possible growth of intermediate entries or guarantee accurate answers for every linear system.^[ref]

To see the mechanism without depending on hardware, define a toy floating-point arithmetic model. The operator ${\mathrm{fl}}_3(z)$ rounds an exact real number to the nearest value with three significant decimal digits; halfway cases choose the even final digit, and ${\mathrm{fl}}_3(0)=0$. Allow an unbounded exponent range. Round the inputs and the result of every addition, subtraction, multiplication and division separately; a product followed by a subtraction rounds twice. This is a prescribed arithmetic model, not a measurement of a physical computer. All inputs in the following original example are represented exactly:Without a row interchange, the elimination multiplier is $m={\mathrm{fl}}_3(1/\epsilon )=10^6$. The last coefficient and right-hand side becomeTwo distinct exact values now have the same rounded representation. Back substitution therefore returnsThe small difference carrying information about $x_1$ was lost during elimination; the final subtraction only exposes that loss. Dividing by the tiny pivot element amplifies sensitivity to the lost information.

With partial pivoting, interchange the two rows, including their right-hand sides, to start from $\left[\begin{array}{ccc}1& 1& 2\\ \epsilon & 1& 1\end{array}\right]$. Now $m={\mathrm{fl}}_3(\epsilon /1)=\epsilon$, and the rounded updates areBack substitution gives ${\hat{x}}_2={\mathrm{fl}}_3(1/1)=1$ and ${\hat{x}}_1={\mathrm{fl}}_3({\mathrm{fl}}_3(2-{\mathrm{fl}}_3(1\cdot 1))/1)=1$. Both paths round intermediate values, but only the first creates million-sized entries and then divides a lost difference by $10^{-6}$.

The true solution follows from the original second equation $x_2=2-x_1$:Evaluate the linear-system residual $r=b-A\hat{x}$ against the original data in exact arithmetic, separately from the three-digit solve. With $\Vert v{\Vert }_{\infty }={\max }_i|v_i|$, the maximum norm, the results areThe respective relative linear-system residuals $\Vert r{\Vert }_{\infty }/\Vert b{\Vert }_{\infty }$ are $1/2$ and $5\times 10^{-7}$. Computing the diagnostic in higher precision matters: a check made only against the rounded reduced equations would miss the original error.

This matrix is well conditioned in the maximum norm: its induced matrix norm is $\Vert A{\Vert }_{\infty }=2$, whileThe failure therefore belongs to this unpivoted algorithm, rather than a large condition number. In general, however, partial pivoting cannot remove sensitivity inherent in the data. For any approximate solution of an invertible matrix system, $\hat{x}-x_{\ast }=-A^{-1}r$; the induced matrix norm inequality and $\Vert b\Vert \le \Vert A\Vert \Vert x_{\ast }\Vert$ giveA small linear-system residual controls solution error only in conjunction with the condition number; numerical stability and sensitivity are different questions.^[ref]

A topological group is a group $G$ equipped with a topology for which both the group operation and taking an inverse element are continuous functions:Here $G\times G$ has the product topology: continuity of multiplication concerns both inputs together. Thus nearby motions compose to nearby motions, and undoing a motion also varies continuously. The definition alone does not impose compactness; we state the Hausdorff space assumption explicitly when it is needed.

The additive group $(\mathbb{R},+)$ of real numbers, with its usual topology, is a topological group: $(x,y)\mapsto x+y$ and $x\mapsto -x$ are continuous functions. Its identity element is $0$. The circle group instead uses multiplication of complex numbers on the unit circle, with the subspace topology from the complex plane:Its identity element is $1$, and its inverse element is the complex conjugate. Multiplication on the complex plane and complex conjugation are continuous functions, so their restrictions verify the required continuity. Angles add, but points multiply: a quarter-turn followed by another quarter-turn gives $i\cdot i=-1$, and the next two quarter-turns return to $1$. Ordinary addition would fail to stay on the unit circle, since $1+1=2$.

For any fixed $a\in G$, the translations $x\mapsto ax$ and $x\mapsto xa$ are homeomorphisms, with inverses given by translation by $a^{-1}$. Consequently the neighbourhoods of the identity element determine the entire topology. This is why local information near one motion can control a whole family of symmetries. See Tornier, Haar Measures, section 1.1 for these basic properties of topological groups.