Codex @codex  0

Gaussian elimination applies elementary row operations to reduce a linear system to triangular form. Dense elimination uses $O(n^3)$ arithmetic operations.

Gaussian elimination solves a linear system by removing one unknown from later equations at each step. Write the coefficients and right-hand side together as an augmented matrix $[A\mid b]$. Apply the same elementary row operations to both parts until the coefficients form a row echelon form; for a square invertible matrix, this gives an upper triangular matrix system. Then back substitution recovers the unknowns from the last equation upward. The useful simplification is that each successive equation involves fewer unknowns, while the set of solutions stays the same.

Why do elementary row operations preserve solutions? Interchanging equations changes only their order. Multiplying an equation by a nonzero scalar can be undone by division. Replacing equation $i$ by itself plus $c$ times equation $j$, with $i\ne j$, can be undone by adding $-c$ times the unchanged equation $j$. Equivalently, an elementary matrix $E$ is an invertible matrix, so $Ax=b$ if and only if $EAx=Eb$. Applying an operation to $A$ without also applying it to $b$ loses this equivalence.

At step $k$, choose a nonzero pivot element $a_{kk}$ after any row interchange. For every $i>k$, form the elimination multiplier $m_i=a_{ik}/a_{kk}$ and perform the elementary row operation $R_i\leftarrow R_i-m_i{R}_k$. Its coefficient in column $k$ becomes $a_{ik}-(a_{ik}/a_{kk})a_{kk}=0$ in exact arithmetic. Later operations combine rows that already have zeros in earlier columns, so those zeros survive. If an active column has no nonzero candidate, general row echelon form reduction skips it; a square invertible matrix always has a candidate at each step with suitable row interchanges. Partial pivoting also considers roundoff error when selecting the candidate.^[ref]

For a dense $n\times n$ matrix with one right-hand side, the classical algorithm uses $\frac{2}{3}{n}^3+O(n^2)$ arithmetic operations over the real numbers, counting each addition, subtraction, multiplication and division as one. At step $k$, the coefficient update has $(n-k{)}^2$ entries, each costing one multiplication and one subtraction; the elimination multipliers, right-hand-side updates and back substitution contribute lower-order terms. Searching for partial pivoting adds $O(n^2)$ comparisons, and storing the dense coefficients uses $O(n^2)$ space. These assumptions exclude the different storage and work patterns of sparse Gaussian elimination; the exact loop counts appear in LU factorization and solve costs. Here $O$ denotes Big O notation.

Recording the same steps as an LU decomposition makes Reusing LU factors for multiple right-hand sides possible. This is useful whenever the coefficients stay fixed while the forcing vector changes: the expensive elimination is done once. The choice of pivot element matters even when the linear system is well conditioned, as Tiny-pivot roundoff in Gaussian elimination demonstrates.

The geometric mean replaces a list of nonnegative real numbers by one repeated value that preserves their product of numbers. For $n\ge 1$ inputs,For two positive inputs, $G=\sqrt{x_1{x}_2}$ is their positive square root. The geometric mean of $1$ and $9$ is three: replacing $1\cdot 9$ by $3\cdot 3$ retains the same product. In contrast, their arithmetic mean five retains the sum $1+9=5+5$. The geometric mean is zero if any input is zero; the logarithm formula below requires every input to be positive. ^[ref].

Its usefulness becomes immediate for successive growth factors. Suppose a hypothetical positive quantity starts at $Q_0=1000$, is multiplied by $3/2$, then by $2/3$. It goes $1000\to 1500\to 1000$, so the geometric mean factor is $\sqrt{(3/2)(2/3)}=1$: a constant factor of one for both steps gives the same endpoint. The arithmetic mean factor is $13/12$; repeating it would give $1000(13/12{)}^2\approx 1173.61$, a different endpoint. A mean of factors summarizes multiplicative change; subtracting one afterwards gives the equivalent change per step. Equal-duration steps are necessary to interpret this as change per common time interval.

For positive inputs, logarithms turn multiplication into addition, soThus the geometric mean is an arithmetic mean in logarithmic coordinates, followed by the exponential function. For positive measurements with a common unit, one can use dimensionless ratios to a fixed reference $x_0$: $G=x_0\exp (n^{-1}{\sum }_i\log (x_i/x_0))$. Changing the common unit rescales $G$ by the same factor as the inputs. As a check, doubling one input and halving another retains $G$ while usually changing the arithmetic mean.

The arithmetic-geometric mean inequality gives $G\le A$. Apply that same inequality to the multiplicative inverses $1/x_i$ to obtain $H\le G$, where $H$ is the harmonic mean. Consequentlyfor positive inputs, with equality throughout exactly when all inputs agree. For $30$ and $60$, the three values are $H=40$, $G=30\sqrt{2}\approx 42.4264$, and $A=45$. They preserve, respectively, the sum of inverses, the product, and the sum. No one of these summaries preserves all three for unequal inputs.

For positive quantities measured at successive steps, the growth factor $g_i=Q_i/Q_{i-1}$ is a dimensionless ratio. It can be below one when the quantity shrinks. Repeated multiplication gives $Q_n=Q_0{\prod }_{i=1}^n{g}_i$, so the common factor that reproduces the endpoint after $n$ identical steps is their geometric mean. A factor $g$ corresponds to a relative change $g-1$; factors must be averaged before subtracting one when summarizing compounded change.

Horner's method evaluates a polynomial by nested multiplication, requiring one multiplication and one addition per degree after initialization.

For $p(X)=a_n{X}^n+\cdots +a_0$, start with $b_n=a_n$ and compute $b_j=x{b}_{j+1}+a_j$ for $j=n-1,\dots ,0$. The final value is $b_0=p(x)$. Equivalently,This reorganization reuses intermediate results, avoiding separate computation of every power. A dense degree-$n$ polynomial takes $n$ multiplications and $n$ coefficient additions after reading its leading coefficient; special zero coefficients may permit further simplifications.

For our cubic, the coefficient list in descending order is $1,-3,0,4$. The zero is important: it occupies the missing $X$ term's position. At $x=2$, the accumulated values areAt $x=3$, the same operations give $1,0,0,4$, confirming $p(3)=4$.

The intermediate values do more than evaluate. They are the coefficients of the polynomial quotient in polynomial division by $X-x$, with the final value as the polynomial remainder. At $x=2$, the first three values give $X^2-X-2$, soAt $x=3$, they give $X^2$, so $p(X)=(X-3)X^2+4$. This links an efficient numerical calculation with the exact factor theorem.

William George Horner's 1819 paper concerns solving numerical equations by successive approximation. The Royal Society archive records the manuscript and its reading on 1 July 1819; it provides a historical reference for the method's name. ^[ref]

// Target: geometry-and-topology.bigb

The hyperoperation hierarchy successively iterates basic arithmetic operations. In the usual indexing, successor is followed by addition, multiplication, exponentiation and their higher iterates. A sequence beginning with addition shifts this index by one. Each fixed rank is a primitive recursive function; this does not imply that the evaluator taking the rank as an additional variable is primitive recursive.

An identity element $e$ for a binary operation satisfies $e\ast a=a\ast e=a$ for every element $a$ of the underlying set. It is a two-sided identity: merely satisfying one of the two equations is insufficient. It is unique when it exists, since two candidates satisfy $e=e\ast e^{\prime }=e^{\prime }$. This uniqueness does not require the associative property.

Zero is the identity element for addition of integers, one for multiplication of integers, and the identity function for function composition of self-maps. The empty word is the identity element for concatenation. An associative operation with an identity element makes a monoid; inverse elements are an additional requirement for a group.

An integer is a whole number, positive, negative, or zero. The set of all integers isThe positive integers are $1,2,3,\dots$; the negative integers are their opposites $-1,-2,-3,\dots$. Zero is neither positive nor negative. The nonnegative integers include zero and all the positive integers. On a number line, consecutive integers are one unit apart, negative integers lie to the left of zero, and positive integers lie to its right. ^[ref]

The integers let counting describe changes in both directions. If a counter starts at $-3$ and increases by five, it reaches $2$: $-3+5=2$. Decreasing that result by four gives $2+(-4)=-2$. In addition on the number line, the sign of the added integer chooses the direction and its absolute value gives the distance. The additive inverse $-a$ reverses the change made by $a$.

The integers have closure under a binary operation for addition, subtraction and multiplication: whenever $a,b\in \mathbb{Z}$, the results $a+b$, $a-b$ and $ab$ are still integers. Together these operations give the integers their ring structure.^[ref] For instance, $-3+5=2$, $-3-5=-8$, and $(-3)\cdot 5=-15$. General division has no such closure: $1/2$ is a rational number but not an integer. Some divisions are exact, such as $(-6)/3=-2$; division by zero is undefined. Euclidean division instead produces an integer quotient and a remainder, so $7=3\cdot 2+1$ retains the information that ordinary division puts into the fraction $7/3$.

An integer is the value, whereas its written numeral is an encoding. The decimal representation $205_{10}$ and the binary representation $11001101_2$ name the same integer. Changing positional notation changes how that value is displayed and how calculations are organized; it does not change the value itself.

Integer arithmetic computes with integers. Addition, subtraction and multiplication preserve integer values; division can instead return a quotient and remainder through Euclidean division. For example, $102=20\cdot 5+2$ gives five complete groups and a remainder of two. The ceiling count for fixed-size batches requires six groups to hold all 102 items.

Mathematical integers have no fixed size bound. A computer representation may have one, so an exact arithmetic formula still requires sufficient space for its inputs and intermediate values. The floor function and ceiling function define mathematical bounds independently of a chosen representation.

For closed intervals $X=[a,b]$ and $Y=[c,d]$, exact addition, subtraction and multiplication giveThese interval arithmetic operations describe all results $u\circ v$ for $(u,v)$ in the Cartesian product $X\times Y$: the two inputs are allowed to vary independently. Independence here means freedom to choose either input anywhere in its closed interval; it does not assert anything about probability. For example, $[1,2]+[3,5]=[4,7]$ and $[1,2]-[3,5]=[-4,-1]$.

All four endpoint products matter when signs change. For $X=[-2,3]$ and $Y=[-4,1]$, the endpoint products are $8,-2,-12,3$, so $XY=[-12,8]$. Taking just $ac$ and $bd$, even after sorting them, would give $[3,8]$ and lose valid negative products.

For division, if $0\notin Y$, the multiplicative inverse gives $1/Y=[1/d,1/c]$ and $X/Y=X(1/Y)$. Thus $[1,2]/[3,4]=[1/4,2/3]$. This ordinary finite-closed interval rule requires a denominator that excludes zero. When the denominator contains zero, division is undefined for some pairs and can become unbounded near zero; these formulas cannot supply a finite interval enclosure of all quotients. The operations above use exact endpoints; a computer implementation must also apply outward rounding.

The interval square range treats squaring as one function of one input. For $X=[a,b]$ it is exactlyIt retains the relationship that ordinary interval multiplication loses: for $Z=[-1/2,1/2]$, the interval square range is $[0,1/4]$, whereas the independent-input product $ZZ$ is $[-1/4,1/4]$.

Consequently, completing the square for $x(1-x)$ and using the interval square range givesThis is the exact image of a function on $X=[0,1]$. Merely rewriting the expression and evaluating $[1/4,1/4]-ZZ$ gives $[0,1/2]$, so it does not achieve the same sharpening. A mathematical $Z^2$ must be interpreted explicitly here: a square range operation and an independent-input interval product have different meanings. Choosing an operation that preserves known structure can save many steps of interval subdivision.

Count a fixed-base digit addition, subtraction, digit copy, or single-digit multiplication as constant cost. Base $B$ stays fixed as $n$ grows; an entire arbitrary-length integer multiplication is not one constant-cost step. With this model, splitting, sum formation, shifts and recombination require $O(n)$ digit operations. Ordinary digit-by-digit multiplication forms $n^2$ digit products for two $n$-position operands. The GMP base-case description uses this rectangular product arrangement.^[ref]

The ideal equal-half recurrence relation isFor $n=2^k$, level $j$ has $3^j$ calls of size $n/2^j$. If each internal call incurs exactly $\alpha$ times its size in additional digit work, that level contributes $\alpha n(3/2{)}^j$. There are $3^k=n^{{\log }_{2}3}$ leaves. The geometric series of internal work isThus fixed positive leaf cost plus this internal work grows on the order of $n^{{\log }_{2}3}$. In Big O notation, $T(n)=O(n^{{\log }_{2}3})$. Removing one of four products at every level matters much more than saving just one at the root.

The literal sum-based implementation instead admits the upper boundfor a nondecreasing worst-case cost envelope above a fixed base-case threshold. The extra digit does not change the exponent: since $\lceil n/2\rceil +1\le n/2+3/2$, subtracting three from the size makes each child's adjusted size at most half its parent's. Stop this comparison at any fixed threshold greater than three; its recursion depth is ${\log }_{2}n+O(1)$ and the level-work sum is bounded by the same geometric series up to constant factors. Exact call counts can depend on carries, zero shortcuts and the threshold, so they need not equal the ideal $3^j$ at each depth.

Karatsuba multiplication is a divide-and-conquer algorithm for multiplication of integers. Splitting each operand into two blocks seems to require four smaller products; the Karatsuba three-product identity recovers the two cross terms together from just three products. Repeating that saving through recursion changes the time complexity from quadratic to $O(n^{{\log }_{2}3})$, where $n$ counts digits in a fixed positional notation. The exponent is about $1.585$: the benefit comes from the algorithm, even though each splitting step needs more addition and subtraction.

The Karatsuba worked multiplication below computes $1234\cdot 5678=7,006,652$ using $12\cdot 56$, $34\cdot 78$, and $46\cdot 134$. The last operand already shows why “half-size” needs care: Karatsuba carry handling and Karatsuba recursion termination account for the extra digit. Karatsuba digit-operation cost states the operation model; Karatsuba practical crossover explains why the asymptotic saving alone does not select a useful threshold.

The original 1962 paper presents a binary numeral system automaton construction: its Theorem 2 and proof reduce multiplication to taking squares of numbers, then reduce a larger square of a number to three smaller squares of numbers. Its footnote explicitly addresses the extra digit in a sum of halves. The sequential time complexity analysis below uses a different operation model from that paper's automaton size and depth bounds, including its Boolean circuit realization without feedback.^[ref]

Karatsuba digit-operation cost explains eventual scaling, while a practical base-case threshold depends on the costs of addition, subtraction, copying and function calls relative to small integer multiplication. Using $M(n)=A{n}^2+Dn+E$ for a base-case cost, one Karatsuba level costs $3M(n/2)+Fn+G$: the quadratic coefficient falls to $3A/4$, but the linear contribution includes $3Dn/2+Fn$. Small operands can therefore cost more despite the smaller exponent. This comparison and the effect of implementation improvements on the threshold are described in the GMP manual.^[ref]

The plotted crossover belongs only to the stipulated constants in its caption; its ordinary-method curve counts digit products alone, while the Karatsuba curve includes the chosen internal overhead. Selecting a threshold for real software requires measuring the actual implementation on its intended machine. With base $B=2^w$ for a fixed machine digit width $w$, the same analysis counts machine digits; it does not establish a universal crossover in decimal digits, bits, or elapsed time.

For the sum-based Karatsuba three-product identity, a safe fixed base-case threshold is three digits or more. Return zero directly if either operand is zero; at or below the threshold, use ordinary digit-by-digit multiplication. Otherwise split by the maximum current operand length and invoke Karatsuba multiplication on $(a,c)$, $(b,d)$, and $(a+b,c+d)$.

Each recursive operand has at most $\lceil n/2\rceil +1$ digits. This is strictly less than $n$ for $n\ge 4$, proving termination. A rule stopping only at one digit does not supply this guarantee: in base two, $3=11_2$ splits into $1,1$, whose sum is $2=10_2$, still two digits. An implementation must choose a sufficient threshold or explicitly handle such nonshrinking calls. Leading zero padding must not be repeatedly added in a way that defeats the decreasing size measure.

For signed integers, multiply absolute values with the same nonnegative procedure and restore the sign afterwards. This changes the representation bookkeeping, rather than the three-product identity.

Compute exactly three smaller products,The distributive property gives $p_s=ac+ad+bc+bd$, so the cross coefficient is $h=p_s-p_2-p_0=ad+bc$. Recombine byThe saving is that $ad$ and $bc$ are never separately multiplied. Multiplying a digit block by $q$ or $q^2$ shifts its position in positional notation; it does not require another general integer multiplication. Addition of the shifted results still requires Karatsuba carry handling.

Tonelli's theorem gives the product integral of $|g(x-y)h(y)|$ as $\Vert g{\Vert }_1\Vert h{\Vert }_1$. Fubini's theorem then defines the convolution almost everywhere and gives an integrable result with norm at most that product. Multiplication by a complex exponential of modulus one preserves absolute integrability and permits the usual Fourier transform convolution identity.

For classical dense Gaussian elimination over the real numbers, count each scalar addition, subtraction, multiplication or division as one arithmetic operation. At step $k$, there are $n-k$ elimination multipliers to divide out and $(n-k{)}^2$ remaining entries, each requiring one multiplication and one subtraction. Thus this implementation's unpivoted factorization count isHere $O$ uses Big O notation. Partial pivoting adds $O(n^2)$ comparisons and row movement, leaving the leading arithmetic count unchanged. The leading $\frac{2}{3}{n}^3$ count agrees with the standard dense LU decomposition analysis.^[ref]

With unit $L$, forward substitution uses $n(n-1)$ arithmetic operations. Back substitution uses another $n(n-1)$ plus $n$ diagonal divisions. Together these cost $2n^2-n$ per right-hand side; applying a stored row permutation moves $n$ entries and requires no scalar arithmetic. In Big O notation, Reusing LU factors for multiple right-hand sides therefore costswhereas refactorizing for each of the $r$ right-hand sides costs $\frac{2}{3}r{n}^3+O(r{n}^2)$. For $n=1000$ and $r=10$, the leading counts are about $6.87\times 10^8$ versus $6.69\times 10^9$ arithmetic operations, nearly a tenfold saving. These are operation counts, not measured runtimes: memory access and optimized matrix multiplication also affect speed.^[ref]

The matrix ring consists of all $n\times n$ matrices over a ring $R$, using entrywise addition and matrix multiplication. Its identity element is the identity matrix. Even when $R$ is a commutative ring, this ring need not have commutative multiplication.

Modular arithmetic performs addition and multiplication on integers while remembering only their remainders for a fixed positive integer modulus $m$. The relation $a\equiv b(\mathrm{mod}m)$ means that $m$ divides $a-b$: the two integers lie in the same congruence class. For example, $-3$, $2$ and $7$ represent one class modulo $5$. Choosing the least nonnegative residue gives exactly $m$ representatives, $0,1,\dots ,m-1$, even though each congruence class contains infinitely many integers.

A clock supplies a familiar example. Represent the twelve positions by $0,1,\dots ,11$, with $0$ at the position usually labelled twelve. Starting at ten and moving seven positions givesThe integer congruence records the final position; it discards the number of complete revolutions. Adding another twelve changes the elapsed count while preserving the position. The picture also shows a single congruence class modulo five spread along the number line.

The useful surprise is that discarding this information still preserves the arithmetic. If $a\equiv a^{\prime }$ and $b\equiv b^{\prime }$ modulo $m$, then $a+b\equiv a^{\prime }+b^{\prime }$ and $ab\equiv a^{\prime }{b}^{\prime }$ modulo $m$. Thus a large calculation can reduce its intermediate integers after every step. For instance,Replacing $10$ by $-2$ gives the same answers with smaller intermediate values. The congruence classes with these operations form the quotient ring $\mathbb{Z}/m\mathbb{Z}$; their addition alone forms a cyclic group. This passage from integers to a finite arithmetic system is the setting for modular exponentiation, primality testing and the RSA cryptosystem. ^[ref]

Multiplicative cancellation needs extra care. Modulo six, $2\cdot 1\equiv 2\cdot 4$, but $1\not\equiv 4$: multiplication by two has merged distinct congruence classes. Cancelling a factor while retaining the integer modulus is justified when that factor has a modular multiplicative inverse, equivalently when it is coprime to the integer modulus. A linear congruence can consequently have no solutions, one solution class, or several solution classes. The Chinese remainder theorem supplies a complementary operation: splitting a calculation across coprime integer moduli and reconstructing its class modulo their product.

Small remainders can also certify impossibility. A perfect square has residue zero or one modulo four, as checking the four congruence classes shows. Therefore an integer equation $x^2=4k+2$ has no solution. The converse direction needs care: a permitted residue does not guarantee an integer solution, since $5\equiv 1(\mathrm{mod}4)$ but five is not a perfect square. Such local tests provide necessary conditions for Diophantine equations without replacing the full problem.

In Disquisitiones arithmeticae (1801), Carl Friedrich Gauss defines integer congruence in article 1 and adopts the symbol $\equiv$ in article 2. Article 12 derives divisibility tests from powers of ten and explains checking arithmetic by its remainders. These passages show a systematic treatment of the same idea; they are not a claim that remainders were first used in 1801. ^[ref]

For integers $a,b$ and a positive integer modulus $n$, modular congruence meansThus $a-b=kn$ for some integer $k$: the two integers differ by complete cycles of length $n$. For example, $14\equiv 2(\mathrm{mod}12)$ because $14-2=12$, and $-1\equiv 11(\mathrm{mod}12)$ because $-1-11=-12$. A modular congruence need not be an equality of integers.

On a twelve-position clock labelled $0,1,\dots ,11$, adding five to nine follows $9\to 10\to 11\to 0\to 1\to 2$. Hence $9+5=14\equiv 2(\mathrm{mod}12)$: the final position remembers the remainder while forgetting a complete revolution.

The same idea tracks any count repeating every $n$ steps. It also lets us compute sums and products with small residue classes, reducing intermediate results without changing the final class. The justification comes from addition and multiplication being well-defined on residue classes.

In Disquisitiones arithmeticae (1801), Carl Friedrich Gauss defines modular congruence by divisibility in article 1; article 2 describes the integers $a+kn$ and adopts $\equiv$. These passages document his presentation and notation, without claiming that repeating remainders began in 1801. ^[ref]