Linear inequality Created 2026-10-14 Updated 2026-10-24
A linear inequality compares a linear functional of a vector of unknowns with a constant, for example or over the real numbers. For , its solution set is a closed or open half-space, respectively. The inequality allows all points on one side of the line , including the line. Multiplication by a negative real number reverses the inequality, so is equivalently .
Linearity of expectation Created 2026-09-24 Updated 2026-10-14
For integrable random variables on the same probability space and fixed real numbers , linearity of expectation states
The random variables can be dependent random variables. The formula follows by the linearity of integration; a finite linear combination of integrable random variables remains integrable. For example, if takes with probabilities , then and are dependent random variables, yet .
An especially useful application counts how many events occur. Put , an indicator random variable, and . Since , we get . There is no need to compute the whole probability distribution of , or to assume independence. For a uniform random permutation of objects, each object stays fixed with probability , so the fixed point count of a uniform random permutation has expected value one for every . [ref].
Linearity of expectation concerns addition and multiplication by fixed scalars. Products instead satisfy under the hypotheses of expectation of a product of independent random variables. For the example above, , whereas . Applying a nonlinear function before averaging can therefore change the answer substantially; use the law of the unconscious statistician to compute that average.
Linearity of integration Created 2026-10-14 Updated 2026-10-20
For integrable real-valued functions on a measure space and fixed real numbers , their linear combination is integrable, and the Lebesgue integral satisfies
The same rule applies to any finite summation: for integrable functions and fixed scalars . Finite signed values matter: integrability ensures all these integrals are finite, so no undefined occurs.
This is the distributive property of finite summation carried into integration: a fixed factor can be pulled out, and a finite sum can be integrated term by term. For simple functions, distribute the finite sums of value times measure; approximation extends additivity to nonnegative measurable functions, and splitting into a positive part and negative part gives the integrable signed case. See Lecture 12: Lebesgue Integrable Functions, the Lebesgue Integral and the Dominated Convergence Theorem (Proposition 122: linearity for integrable functions). For example, . With a probability measure, linearity of integration gives linearity of expectation.
Loop variant 2026-10-18
A decreasing measure used to prove algorithm termination. A nonnegative integer loop variant must strictly decrease after every completed body whose condition was true; the body itself must terminate.[ref]
The prefix maximum correctness example uses . It counts unread entries and is independent of whether the new entry changes the maximum. Ties therefore cannot stall progress: increases even when stays the same. An integer bounded below cannot strictly decrease indefinitely, which supplies the missing algorithm termination argument.
Figure 1.
Control flow and four proof obligations for the prefix maximum correctness example
. Initialization, preservation, and exit establish partial correctness; the loop variant establishes algorithm termination and hence total correctness. The loop invariant is restored after both variable assignments. The input array stays unchanged throughout. Details.
Strict decrease and a lower bound are insufficient for arbitrary real numbers. The positive sequence strictly decreases and is bounded below by zero, but never reaches zero. Choosing an integer measure avoids this failure.
Low-rank approximation 2026-10-14
A low-rank approximation replaces an matrix by a matrix of matrix rank at most a chosen integer , with , while making an explicitly chosen error norm small. The point is to describe many entries using a few shared patterns: an image compression may retain broad brightness patterns while discarding finer ones. The approximation is useful when the saved storage or computation justifies the lost detail. A small matrix rank alone says nothing about the error.
The compact representation is a sum of outer products, equivalently a product of two thin matrices:
Here are the columns of , and the star denotes the conjugate transpose; over the real numbers use the transpose. The rank bound for a matrix product gives . Conversely, choose a basis of the column space of as the columns of , and put the coordinates of each column of into ; pad with zero columns if the matrix rank is smaller than . Each nonzero outer product is a rank-one matrix: it repeats a single column pattern at different strengths across the columns.
Keeping the two factors stores scalars instead of . This saves space only when . The usual singular value decomposition representation stores scalars because it keeps the singular values separately; absorbing them into one factor removes that extra array. These are representation sizes at equal scalar precision, before file overhead. See Uschmajew and Vandereycken, §9.2: thin factors and their storage, and Illinois CS 357, Low-rank approximation: storing singular vectors and values.
The factors also give without forming the full matrix . Direct evaluation uses scalar multiplications, compared with for a dense matrix applied to a vector. For , the two factors contain scalars rather than , and the same counts describe these multiplication costs. Constructing the factors is a separate cost. Moreover, a low-rank approximation need not be a sparse matrix: the constant matrix whose entries are all has matrix rank despite having no zero entries.
For the standard unweighted problem, write a singular value decomposition with and singular values . Retain the first paired left singular vectors and right singular vectors:
The Eckart-Young-Mirsky theorem says that this same minimizes both of the following error norms over all matrices with matrix rank at most :
where and an empty sum is zero. Here the Frobenius norm is , whereas the operator norm is specifically induced by the Euclidean norm, . The first aggregates errors in all entries; the second bounds the worst output error for a unit vector input. These formulas apply to real and complex matrices. [ref]
For , a relative Frobenius norm tolerance is met precisely when the discarded squared singular values sum to at most . Thus one can choose the smallest suitable before reconstructing the matrix. Many individually small discarded singular values can still have a large combined error. An operator norm tolerance instead tests only the largest discarded singular value, . The two questions can therefore select different ranks.
The following stipulated matrix makes the tradeoff visible. Define four real vectors of length :
The vectors are orthonormal, so this is already a singular value decomposition written as outer products, with positive singular values . All entries lie between and and can be displayed as brightnesses. The first component is uniform brightness, the second distinguishes broad blocks, and the last two supply finer alternating patterns. This is a designed arithmetic example, not measured image data.
Figure 1.
Shared patterns in a stipulated matrix
. The first three panels use the same brightness scale. Rank one retains uniform brightness; rank two adds the broad blocks. The signed residual has its own color scale to expose the discarded patterns. The factors apply a linear map through just intermediate coordinates. Details.
Here has every entry equal to , while has entries within the two diagonal blocks and in the other two blocks. For instance, becomes : its error is . The full residual is , whose Frobenius norm is and whose operator norm is . Since , the relative Frobenius norm error is . Two thin factors use scalars, or if the normalized left singular vectors, right singular vectors, and singular values are stored separately, instead of dense entries.
Figure 2.
Error falls as storage rises for the stipulated matrix
. The exact Frobenius norm and operator norm errors are plotted for . The storage panel counts the two thin factors, or the normalized left singular vectors and right singular vectors plus separate singular values, at equal scalar precision. Lines join discrete rank choices; they do not represent intermediate ranks or measured data. Details.
Retaining three components reduces the Frobenius norm error to while using scalars in two factors. Retaining all four reconstructs exactly but uses scalars, giving no saving in this generic representation; keeping the singular values separately uses . Our particular sign patterns could be encoded more economically, but that would exploit an additional structure beyond the general low-rank approximation. The striking feature is that numerical error and storage can both be read from the decomposition, rather than judged only by how smooth the picture looks.
The constraint is deliberately matrix rank at most . The truncation has . If , the best approximation is itself, with zero error. Requiring matrix rank exactly can destroy attainment: for and , the matrices have matrix rank for , and both error norms equal . Their error has infimum zero, but no rank-two matrix attains it. If , the displayed does have matrix rank exactly and attains the same minimum under either constraint.
Optimality also does not always mean uniqueness. For , the best Frobenius norm approximation is unique when . [ref] A positive tie across the cutoff permits different choices: for and , both and have Frobenius norm error and operator norm error . More generally, is optimal for every real unit vector in the first two coordinate directions. Rotating which tied direction is kept changes the approximation. Repeated singular values entirely inside the retained group do not change its sum of outer products. At the only feasible matrix is zero; at the unique zero-error approximation is , even if some singular values are tied at zero.
A strict gap does not guarantee uniqueness for the operator norm. For and , every with has matrix rank and error , which is optimal. Only minimizes the Frobenius norm, since . An unchanged worst direction can hide extra error in another direction under the operator norm.
Finally, these guarantees concern the stated norms and the sole matrix rank constraint. They do not establish that a discarded pattern is noise or irrelevant to a particular task: the finer patterns in our constructed example were put there intentionally. Adding entrywise weights or restrictions on the factors changes the problem, so the Eckart-Young-Mirsky theorem above does not automatically solve it. For historical context, the original 1936 matrix problem is documented in the existing Eckart-Young-Mirsky theorem section.
For classical dense Gaussian elimination over the real numbers, count each scalar addition, subtraction, multiplication or division as one arithmetic operation. At step , there are elimination multipliers to divide out and remaining entries, each requiring one multiplication and one subtraction. Thus this implementation's unpivoted factorization count is
Here uses Big O notation. Partial pivoting adds comparisons and row movement, leaving the leading arithmetic count unchanged. The leading count agrees with the standard dense LU decomposition analysis.[ref]
With unit , forward substitution uses arithmetic operations. Back substitution uses another plus diagonal divisions. Together these cost per right-hand side; applying a stored row permutation moves entries and requires no scalar arithmetic. In Big O notation, Reusing LU factors for multiple right-hand sides therefore costs
whereas refactorizing for each of the right-hand sides costs . For and , the leading counts are about versus arithmetic operations, nearly a tenfold saving. These are operation counts, not measured runtimes: memory access and optimized matrix multiplication also affect speed.[ref]
Lusin set 2026-10-06
A Lusin set is an uncountable subset of the real numbers such that is countable for every meagre set . Every meagre subset has a meagre cover, so testing those covers suffices. The Lusin set construction under CH gives an example. A Lusin set has no uncountable meagre subset of itself.
Luzin set 2026-10-06
An uncountable subset of the real numbers with countable intersection with every meagre set. This is the category counterpart of a Sierpiński set.
Matrix proportional to its transpose Created 2026-10-20 Updated 2026-10-25
For a matrix over the real numbers or complex numbers, applying the transpose twice to gives . A nonzero matrix is therefore possible only for or . These cases are precisely the symmetric matrices and skew-symmetric matrices, respectively. For every other scalar, the only solution is the zero matrix.
Matrix rank Created 2026-09-24 Updated 2026-10-18
The matrix rank of an matrix over a field is the dimension of its column space, equivalently the dimension of its row space. It counts linearly independent column directions, with . Counting nonzero matrix entries gives a different quantity: an all-ones matrix can have arbitrarily many nonzero matrix entries and still have matrix rank one.[ref]
Consider the following matrix over the real numbers, with columns written as vectors:
The vectors are linearly independent: requires both and , so . The third column adds no new direction to their linear span, hence despite all nine matrix entries being nonzero. The second row likewise repeats twice the first row.
The column space is a plane through the origin:
Every output has second coordinate twice its first coordinate. Conversely, any vector equals , so this condition describes the entire plane, not just part of it. For example, is attainable, whereas lies outside the plane.
Figure 1.
Three column vectors span a two-dimensional plane: , is attainable, and is outside the column space
. Details.
Elementary row operations expose the same count algebraically. Apply and , interchange the last two rows, multiply the second row by , and subtract twice the second row from the first. The resulting reduced row echelon form is
There are two pivot columns, so the matrix rank is two. The corresponding original columns form a basis of the original column space. The pivot columns of itself span the different plane with third coordinate zero: row operations preserve column linear relations, but generally move the column space.[ref]
Solving the homogeneous linear system gives and . Taking the free variable gives
The kernel is the input direction that trades one unit of each of the first two columns for one unit of the third. This trade leaves the output unchanged. The rank-nullity theorem expresses the general balance: for an matrix of matrix rank , the nullity is . The column space describes reachable outputs; the kernel describes invisible changes to inputs.[ref]
For the attainable output , one solution is . Every solution lies in the affine solution space of a linear equation
Indeed, and . Conversely, subtracting from any solution gives a vector in the kernel, so this describes every solution. For , the same row operations on the augmented matrix expose : the second requirement demands where twice the first requires .
Figure 2.
Two pivot columns, one kernel direction, and the same row operations distinguishing attainable from impossible
. Details.
In general, a linear system is solvable exactly when appending leaves the matrix rank unchanged:
Appending a vector outside the column space adds an independent direction. When a solution exists, every solution is , because . Thus one count predicts both how many output directions are reachable and, through the rank-nullity theorem, how many independent input directions remain free.[ref]
Matrix trace Created 2026-09-24 Updated 2026-10-14
For an -by- square matrix over a field, its matrix trace is the sum of its diagonal entries:
For example, has trace . Adding corresponding diagonal entries immediately proves linearity:
Thus matrix trace is a linear functional on the vector space of -by- matrices.[ref]
The surprising point is that the sum is intrinsic to a linear operator, even though individual diagonal entries depend on the basis. The cyclic property of the trace gives, for any invertible matrix ,
Hence matrix similarity preserves trace. For the example above, choose the change-of-basis matrix . Then
The diagonal entries have changed but their sum has not. This makes trace useful for extracting a feature of the underlying linear operator without finding a preferred basis.
Over the complex numbers, or over the real numbers after allowing complex eigenvalues, the trace equals their sum with algebraic multiplicity:
No diagonalizable matrix hypothesis is needed.[ref] To see why, use triangularization over an algebraically closed field: an upper triangular matrix similar to has the eigenvalues on its diagonal, counted with algebraic multiplicity, because its characteristic polynomial is . Similarity preserves trace, so summing that diagonal proves the formula.
For instance,
There is only one independent eigenvector, since , so is not a diagonalizable matrix. Nevertheless its repeated eigenvalue contributes twice. For our earlier , the characteristic polynomial is , its eigenvalues are and , and their sum is . An inexpensive diagonal sum therefore gives spectral information before solving the characteristic polynomial.
The trace of a real matrix does not give that linear map's volume factor: that factor is the absolute value of its determinant. For example, has trace and area factor , but has trace and volume factor . Their occasional numerical agreement in two dimensions is a coincidence. The connection between trace and local volume change gives a geometric interpretation near the identity matrix, and describes the volume rate of a flow map generated by a linear system of differential equations.
Matroid greedy algorithm 2026-10-14
Assign each element of the finite ground set a real number . The matroid greedy algorithm sorts elements by nonincreasing weight, starts with the empty set, and admits an element exactly when the enlarged selection is a matroid independent set. Ties can be resolved arbitrarily. After all elements have been considered, the result is a matroid basis of maximum total weight among all matroid bases. This holds even with negative weights: the task is to produce a full matroid basis. [ref]
Maximizing weight over all matroid independent sets is a different task when weights can be negative. For that task, skip negative elements; zero-weight additions are optional. With weights , , in the weighted three-vector matroid, the heaviest matroid basis is with weight , while the heaviest matroid independent set is with weight . For nonnegative weights, every matroid independent set can be extended to a matroid basis without reducing weight, so the optimum values coincide.
Proof outline. Let be the admitted elements in order, and order any competing matroid basis by nonincreasing weight. If , exchange would let one of augment . That element was considered before and, by heredity, was admissible even at that earlier stage, a contradiction. Hence for every , and summing proves optimality. Rejected elements remain inadmissible as the selection grows, by heredity, so the final selection is maximal. The guarantee depends on the matroid independence axioms and on weights adding independently across elements; interactions between chosen elements are a different objective.
Maximum subarray problem 2026-10-20
Given a finite sequence of real numbers, find a nonempty contiguous block with maximum sum. Its restriction to contiguous blocks distinguishes it from choosing an arbitrary subset. The problem admits a linear-time solution using Kadane's algorithm.
Measure theory Created 2026-09-24 Updated 2026-10-14
Measure theory studies measures, which assign sizes to measurable sets, and the Lebesgue integral, which accumulates values weighted by those sizes. The same framework describes Lebesgue measure on the real numbers, counting measure on discrete collections, and probability measures on possible outcomes. The choice of measure determines what counts as large; the choice of sigma-algebra determines what can be measured.
A striking example separates size from proximity: the rational numbers in form a dense set with Lebesgue measure zero, although the whole interval has Lebesgue measure one. This distinction makes almost everywhere statements useful: exceptional null sets can be ignored in a Lebesgue integral even when they meet every open subinterval. The monotone convergence theorem and dominated convergence theorem then give precise conditions for passing a limit through an integral.
A historical starting point is Lebesgue's 1902 thesis, [ref]. Its title connects integration with the geometric task of assigning sizes.
Metric Created 2026-09-24 Updated 2026-10-14
A metric on a set is a function satisfying, for every :
A metric equips its underlying set with a metric space structure; the distance function and the space carrying it are distinct concepts. The second requirement distinguishes a metric from a pseudometric, which may identify distinct elements at zero distance. The symmetry requirement also explains why directed travel times, such as uphill versus downhill times, need not define a metric.
A tempting alternative on the real numbers is . It is nonnegative, symmetric, and zero exactly on equal endpoints, but fails the triangle inequality: . Checking only the first three requirements would miss the failure.
Minimal polynomial of a normal operator Created 2026-10-25 Updated 2026-10-31
On a real or complex inner product space of finite dimension, a normal operator has a square-free polynomial as its minimal polynomial. To prove this without the spectral theorem, suppose an irreducible polynomial occurs at least twice in . Put and . Since polynomials of a normal operator are normal, is a normal operator. For every ,
By kernel stabilization for a normal operator, . Hence the smaller-degree monic polynomial annihilates , contradicting minimality.
Over the complex numbers, all irreducible polynomials are linear, so the primary decomposition theorem for linear maps now gives a basis of eigenvectors. Over the real numbers, irreducible quadratics can remain: a nontrivial planar rotation need not have any real eigenvectors, although its minimal polynomial is square-free.
Minkowski inequality Created 2026-10-05 Updated 2026-10-14
For measurable functions with values in the real numbers or complex numbers on the same measure space , and , membership in the same Lp space gives
Thus adding two functions cannot make their combined Lp norm exceed the sum of their separate Lp norms. The measure need not be finite or a probability measure. The hypotheses require finite Lp norms; uses the essential supremum of the absolute value, not an ordinary pointwise supremum. For complex numbers, the absolute value is the complex modulus. As usual, Lp spaces identify functions equal almost everywhere. [ref]
The Minkowski inequality makes the Lp norm a workable notion of size: it supplies the triangle inequality needed to turn into a metric on Lp space. Applied to , it says that travelling through an intermediate function cannot shorten the direct distance measured by this metric. Applied to two error functions, it gives a bound on the total error even when the errors reinforce one another; Minkowski inequality for error aggregation gives explicit examples.
The geometry is already visible in the Euclidean plane. Under counting measure on two coordinates, the closed unit balls for the L1 norm, Euclidean norm, and L-infinity norm are respectively , , and . Every one is a convex set: if have Lp norm at most one, the Minkowski inequality and absolute homogeneity of a norm give for . The Euclidean norm has no flat boundary segments, whereas the other two norms do. This is the distinction captured by a strictly convex normed space and developed in equality in Minkowski inequality.
Figure 1. . The red point is the midpoint of the two marked unit vectors. It lies inside the disk, but on the flat edge of the diamond or square. All three unit balls are convex sets; only the disk gives a strictly convex normed space among these planar examples. Details.
A short Hölder's inequality proof outline handles . First , so has finite Lp norm before any division. With conjugate exponents and , set ; then . The scalar triangle inequality followed by Hölder's inequality gives
Divide if ; otherwise the desired bound is immediate. For , integrate . For , the almost everywhere bounds and give the result by the essential supremum. The same scalar triangle inequality works for real numbers and complex numbers.
Let be an integer not divisible by . Its modular inverse modulo has a unique least nonnegative representative . Reducing the inverse equation modulo shows that the representatives are compatible. Since is a p-adic unit,
so the sequence converges to the inverse in . This computes a fraction through finite integer operations.
For and , the first eight representatives are . Each satisfies . The resulting expansion is
The geometric series has ratio , whose 2-adic absolute value is , and its sum is . In the ordinary real number absolute value, the representatives grow. Different metrics impose different meanings of a good approximation. The periodic digits here are a specific instance of the rational expansions studied in Conrad's authored note.[ref]
Figure 1.
Finite modular inverses approximate one third in the 2-adic metric
. Original exact least nonnegative inverses of modulo . The left panel plots and the bound . The right plots the ordinary real error . Equal neighbouring representatives arise when the next binary digit is zero, so their exact errors also agree. Details.
Monotone subsequence 2026-10-25
A monotone subsequence is a subsequence which is a monotone sequence. The indices always increase strictly, but equal selected values are permitted. The monotone subsequence theorem guarantees one for every sequence of real numbers.
For a sequence of real numbers, a common finite limit of a sequence for all monotone subsequences forces the whole sequence to have that limit of a sequence. There is no need to assume a bounded sequence if the hypothesis already includes convergence of every monotone subsequence.
If fails to tend to , select a subsequence with for some fixed . The monotone subsequence theorem gives a monotone subsequence of this subsequence, which contradicts the common limit of a sequence . Conversely, every subsequence of a convergent sequence has the original limit of a sequence.
For a bounded sequence, the bounded monotone sequence theorem already ensures that all monotone subsequences converge. Only agreement of their limits of sequences remains to be checked.