Adjoint operator Created 2026-09-24 Updated 2026-10-14
For a bounded linear operator between real or complex Hilbert spaces, its adjoint is the unique bounded linear operator satisfying
The inner product here is linear in its first argument. The adjoint operator transfers an output measurement back to the input: measuring against the vector gives exactly the same scalar as measuring against . It depends on both inner products.
Figure 1.
The adjoint transfers a measurement direction while preserving its measured value
. Details.
In this real example,
Both inner products equal . The dashed lines collect all vectors giving that same measurement: the output line becomes the input line . The adjoint operator therefore moves the direction that describes the measurement. It generally differs from the inverse matrix: here .
For an explicitly complex example, use the standard inner product on . In the standard orthonormal basis, the adjoint operator is represented by the conjugate transpose:
For and , direct matrix multiplication gives
and hence
The complex conjugate is essential to this agreement.
Existence follows from the Riesz representation theorem: for fixed , the bounded linear functional has a unique representing vector . The resulting adjoint operator has the same operator norm as , and
MIT's lecture 18 develops the representation argument and the conjugate transpose example.[ref]
This transfer explains the normal equation for a linear inverse problem: a least-squares solution has residual orthogonal to every attainable direction , so for every , equivalently . An output residual becomes an input condition through the same adjoint operator. All identities in this section concern bounded operators defined on the whole source Hilbert space; for a densely defined operator that is not bounded, use the adjoint of a densely defined operator and check its operator domain.
Algebra Created 2026-09-24 Updated 2026-10-13
Algebra studies equations, algebraic operations, and the structures that make their rules work. Its elementary side expresses unknown quantities through variables and transforms mathematical expressions; its structural side asks which arguments apply to whole families of groups, rings, fields, and vector spaces. The useful step is identifying exactly which properties a calculation needs, so that it can be reused in a new setting.
For example, the polynomial equation over the real numbers factors as , giving roots and . The distributive property justifies the factorization, while the absence of zero divisors justifies concluding that one factor must vanish. Over the finite field , the same integer coefficients reduce to , with roots and . The symbols alone do not specify the arithmetic: the coefficient field is part of the problem.
In a ring with zero divisors, a familiar step can fail. Modulo , all four residue classes satisfy . For , the factorization holds modulo , yet neither factor is zero. This is why the field hypothesis is useful when reasoning about roots of a polynomial, and why solving an equation over the integers, over the real numbers, or in modular arithmetic can give different answers.
The structure is also useful beyond scalar arithmetic. Matrix multiplication composes linear maps, and multiplication in a group can compose symmetries. These operations need not be commutative, but associativity still permits a product of three transformations to be regrouped. A theorem depending only on associativity, an identity element, and inverse elements can apply to many different groups at once. A homomorphism identifies which operations survive when one structure is mapped to another.
Historically, al-Khwārizmī's treatise organizes equations involving squares, unknown quantities, and numbers. Frederic Rosen's 1831 English translation includes the calculation now written and geometric explanations for completing the square. In modern notation, gives or ; the historical treatment seeks the positive quantity . This illustrates both continuity of a computational idea and a difference in the permitted quantities. [ref]
For dimension-compatible matrices, . In coordinates both sides have entry ; distributing and regrouping the finite sums proves equality. This permits accumulating a product left to right even when a recursive algorithm groups the same transformations from the right. It does not permit changing factor order, because matrix multiplication is generally not commutative.
For a diagonal matrix over a field, matrix multiplication gives . Hence a matrix commutes with exactly when wherever . If all diagonal entries are distinct, the centralizer of a diagonal matrix consists exactly of diagonal matrices. When entries repeat, the allowed entries form unrestricted blocks on the corresponding eigenspaces.
Closed walk Created 2026-10-06 Updated 2026-10-14
A closed walk is a walk with equal endpoint vertices, . Under our convention this includes the zero-length walk ; returning after traversing at least one edge requires . Closure imposes no other restriction on repeated vertices or edges. In a directed graph, a closed directed walk must also follow every directed edge in its permitted direction.[ref]
Figure 1.
Returning to : backtracking closed walk, closed trail with a repeated internal vertex, and graph cycle
. Details.
In the shared simple graph, is a length-two closed walk that immediately backtracks along . It is not a trail, since is used twice, and it is not a graph cycle. The length-six closed walk is a trail, since its six edges are distinct, but is not a graph cycle: it returns to midway through the journey. The length-three closed walk is a graph cycle, with all vertices distinct apart from the required endpoint repetition. In a simple graph that is an undirected graph, a graph cycle has length at least three; the two-step backtrack is excluded.[ref]
A positive-length closed walk therefore need not reveal a graph cycle: even a tree permits an out-and-back journey on an edge. A positive-length closed trail in a simple graph does contain a graph cycle. For a proof outline, choose a shortest positive-length consecutive portion of the trail whose endpoint vertices coincide. It cannot repeat an internal vertex, since that would give a shorter such portion; it cannot have length one or two because self-loops and repeated edges are excluded. It is consequently a graph cycle. Cutting it out leaves another closed trail, possibly of length zero, so repetition partitions the original edges into graph cycles. In our six-step example these are the two triangles in a graph. A closed trail using every edge would be an Euler circuit; here is omitted.
Counting return journeys makes closed walks useful even when they repeat edges. For a finite simple graph with adjacency matrix , the walk count from powers of an adjacency matrix gives length- walks from to , for every nonnegative integer . A proof outline is that matrix multiplication appends one allowed step, summing over the previous endpoint; is the identity matrix, recording the unique zero-length walk at each vertex. Consequently
The trace counts closed walks with a specified starting vertex and traversal order, not distinct graph cycles. These formulas use adjacency entries, not arbitrary travel costs in a weighted graph.[ref]
In our example, : the five length-two closed walks are , , , , and . Also , going round either triangle in a graph in either direction, and : each of the two triangles in a graph supplies three choices of starting vertex and two directions. These exact counts for the constructed graph illustrate why return counts and graph cycle counts differ.
A closed walk naturally models a delivery or inspection journey that returns to its depot. Repeated vertices let it combine several excursions; repeated edges let it serve a dead end and return. For an Euclidean travelling salesman tour, one can first construct a closed walk visiting all locations and then use the triangle inequality to shortcut repeated visits without increasing total length. That argument assumes direct travel between the retained locations is permitted, as in a complete graph weighted by Euclidean distance; an arbitrary road network need not contain those shortcut edges.
Column space Created 2026-09-29 Updated 2026-10-18
The column space of a matrix is the linear span of its columns. It is the image of a linear map represented by .
If an matrix has columns , then matrix multiplication gives . Thus its column space consists of exactly the attainable output vectors in , and has a solution exactly when belongs to this column space. Its dimension is the matrix rank; the kernel instead describes input vectors in whose contributions cancel.[ref]
Commuting elements 2026-10-14
Two elements commute when their two products in opposite orders agree. A particular pair of matrices may commute even when matrix multiplication is not a commutative operation on the whole set of matrices.
Covariance matrix Created 2026-09-24 Updated 2026-10-14
For a real random vector with finite second moments, its covariance matrix collects all pairwise covariances:
The diagonal entries are variances. The covariance matrix is a symmetric matrix and a positive semidefinite matrix, because for every real vector ,
Thus one matrix describes the variance of every linear combination of the coordinates. It is a positive-definite matrix exactly when no nonzero linear combination is constant almost surely. A deterministic linear relation produces a zero eigenvalue.
For example,
has unit coordinate variances, covariance , and eigenvalues and . Its unit eigenvectors and describe directions of greatest and least projected variance. For a unit vector , direct multiplication gives : the variation in the sum direction is nine times that in the difference direction.
Figure 1.
A covariance matrix encodes projected variance in every direction
. Left: the covariance ellipse has a semi-major axis of length and a semi-minor axis of length along the eigenvectors of . Right: the variance of the unit-direction projection is . This is exact matrix geometry for the stipulated covariance matrix, with no sampled observations or claimed probability coverage. Details.
The spectral theorem supplies an orthonormal basis of these directions. This connects covariance matrices to principal component analysis: its leading eigenvector identifies the unit-direction projection with the largest variance. For a linear transformation , the transformed random vector has covariance matrix , so uncertainties propagate through a linear model by matrix multiplication.
Determinant Created 2026-09-24 Updated 2026-10-14
The determinant assigns a scalar to a square matrix. For a linear map on the real plane, it measures signed area scaling: its absolute value tells how much area grows, and its sign records whether the orientation of a vector space is preserved or reversed. In three dimensions, the same idea measures signed volume. Over a field, a square matrix is an invertible matrix exactly when its determinant is nonzero. These are two views of one phenomenon: losing an independent direction collapses full-dimensional volume and makes the original input impossible to recover uniquely.[ref]
Start with an explicit matrix and its action on a vector:
The columns are the images of the standard basis vectors: and . Consequently the unit square with corners becomes the parallelogram with corners . Its area is five. Every small patch of this square undergoes the same linear map, so the factor describes the whole deformation, rather than just one specially chosen boundary. The matrix mixes horizontal and vertical components, but its determinant compresses that combined effect into one area factor.[ref]
Now exchange the columns:
The parallelogram occupies exactly the same region, but its generating vectors arrive in the opposite order. Moving from the first column to the second now turns clockwise rather than counterclockwise. Ordinary area remains five; the oriented volume in two dimensions changes sign. A negative determinant therefore signals reversed orientation of a vector space, not a negative amount of material. Keeping the sign makes successive reversals calculable: two reversals restore the original orientation of a vector space.
Compare a singular matrix:
Its columns are linearly dependent: the second is twice the first. The unit square collapses onto the segment from to , which has zero area despite nonzero length. Since , the kernel of a linear map contains a nonzero vector; inputs differing by that vector have identical outputs. The collapsed picture explains why an inverse matrix cannot exist.
Figure 1.
The unit square, determinant five, reversed orientation, and collapse to zero area
. Details.
For , recovery is possible, and the complete matrix multiplication check is
Here is the identity matrix. The inverse matrix restores the original square, shrinking its area by . Generally, the adjugate identity gives whenever the determinant is nonzero over a field; this is why the denominator matters.[ref]
The multiplicativity of the determinant describes two successive deformations. Matrix multiplication means that applies first, then . Take
The first linear map doubles area; the second multiplies the resulting area by five. Thus the unit square passes through areas , with the last parallelogram bounded by . Direct calculation agrees:
This explains physically why area factors multiply instead of adding. The general identity holds for square matrices of the same size, including singular matrices. If either stage loses a direction, later processing cannot recover the discarded information.[ref]
Figure 2.
Successive linear maps multiply area factors: one becomes two, then ten
. Details.
The same interpretation works in three dimensions without drawing a new picture. The diagonal matrix
sends the unit cube to a rectangular box with side lengths two, three, and one. Its ordinary volume is six. The minus sign comes from reversing the third direction, so the oriented volume is minus six. Replacing the last diagonal entry by zero flattens the cube into a planar region: its area may be positive, but its three-dimensional volume is zero. This distinction matters whenever a linear map changes shape: the determinant measures size in the full input dimension, rather than the size of whichever lower-dimensional object remains.
The determinant measures neither trace nor perimeter. For , the trace happens also to be five, but the identity matrix in two dimensions has trace two and determinant one. The first parallelogram above has perimeter , compared with four for the unit square; boundary length follows a different rule. Nor does determinant one require every length to stay fixed: the diagonal matrix preserves area while stretching one direction and compressing the other.
These properties make the determinant useful when coordinates obscure what a deformation does. A nonzero determinant certifies that linearly independent input directions remain distinguishable; its magnitude counts the area or volume represented by a transformed cell. That second role reappears in the change of variables formula, where the Jacobian determinant corrects the sizes of small cells. Signed volume also separates a deformation that reverses the orientation of a vector space from one that merely stretches it, even when both give identical ordinary volume factors.[ref]
For general , regard the columns as ordered arguments. The determinant is the unique alternating multilinear map to the scalar field normalized by . The multilinearity of the determinant means linearity in each column separately, with all others held fixed:
Alternation means equal columns give zero; exchanging columns negates the result. Normalization sets the oriented volume of the standard basis to one. These properties also hold in rows. For real three-dimensional columns, their determinant is the scalar triple product, and its absolute value is the enclosed volume. Over other fields, the algebraic definition still works even without a geometric notion of positive volume.[ref]
Separate linearity does not mean the determinant is a linear map on the whole matrix. Scaling every column of an -by- matrix by gives . For our two-dimensional example, : doubling both directions quadruples area. The power records how many independent directions are being scaled together.
The properties suggest practical computations. Adding a multiple of one column to another preserves the determinant: the extra term has repeated columns and vanishes. This is the algebra behind an area-preserving shear mapping. Alternatively, expansion along a row uses cofactors, each a matrix minor multiplied by a sign:
where deletes row and column . The Leibniz formula for determinants supplies the general entry formula. Both calculations encode the same geometric invariant; the defining properties explain their structure without requiring memorization of a separate recipe for every size.[ref]
Consider the original linear system
Completing the square proves that is a symmetric positive-definite matrix:
The Jacobi preconditioner has . Symmetric positive-definite preconditioning gives
Its eigenvectors and have eigenvalues and . Hence
The characteristic polynomial of is . Its eigenvalues and spectral condition number of a positive-definite matrix are
The ellipse axis ratio drops from about to . The transformation multiplies the second coordinate by ten, bringing the two energy scales together.
Figure 1.
Diagonal scaling changes thin energy ellipses into nearly circular energy ellipses
. Details.
Both panels show the same energy level sets, expressed in error coordinates and . Their coordinate axes have equal scales. The marked displacement becomes ; its energy remains . These are exact contours of the specified quadratic form.
Solving the original equations gives
Direct matrix multiplication checks both linear-system residuals:
By contrast, left preconditioning gives , which is visibly not a symmetric matrix, despite having the same positive eigenvalues as .
A bad scaling can reverse the benefit: the still valid symmetric positive-definite matrix gives . Its Rayleigh quotients on the coordinate unit vectors are and , so . Cheapness and positive definiteness do not guarantee improved conditioning.
Exponentiation by squaring Created 2026-10-03 Updated 2026-10-20
Exponentiation by squaring computes an integer power by reading the exponent's binary expansion. Appending a binary digit to a prefix of value replaces by , so the corresponding power changes by
Each new digit requires one squaring and, only for digit one, one further multiplication. The number of digits grows with the logarithm of the exponent, making this useful for modular exponentiation, primality testing, and powers of matrices using matrix multiplication.[ref]
For a concrete calculation, , so . The optimized left-to-right algorithm used here instead keeps one accumulated power: initialize from the leading digit one, then process only the remaining digits :
Thus , using three squarings and two further multiplications, instead of twelve successive multiplications starting at .
Figure 1.
Reading the same binary expansion computes and
. Each appended binary digit changes the prefix value by . The ordinary-power row grows to ; the modular exponentiation row stores only remainders . Both traces use three squarings and two further multiplications. Details.
For a general positive integer with leading digit , set and, for , first replace by , then multiply it by if . Return . This explicitly skips the leading digit's otherwise unnecessary operations on one. A Computational Introduction to Number Theory and Algebra by Victor Shoup (Section 3.4, pp. 65–66) presents the version initialized at one and processing every digit; the optimization here makes the counting convention unambiguous.
The loop invariant is , where is the value of the digits already read. Initially and . If the next digit is , then squaring and optionally multiplying give , exactly the next prefix. At termination . This proof uses associativity of multiplication; it works in any monoid, including matrices with the identity matrix as identity element, and needs no commutativity between different bases.
For , return the identity element without performing a multiplication. This is the empty-product convention ; if , the algorithm therefore returns one for by convention, rather than resolving its meaning in every mathematical setting. Negative integer exponents require an inverse element: compute , counting the inversion separately. Over the integers, such an inverse generally leaves the integers, and a zero base has no inverse.
Farkas' lemma Created 2026-10-07 Updated 2026-10-14
For a real matrix and a vector , exactly one of the following systems has a solution:
The linear inequalities here are componentwise. In the first system is nonnegative; in the second system is unrestricted in sign. The second system is a certificate that the first has no feasible point. This nonnegative-equations alternative is developed in Boyd and Vandenberghe, Convex Optimization slides, §5.34.
Build the target, or separate it. Write the columns of as . Then is a conic combination: its coefficients must be nonnegative, but need not sum to one. The obtainable targets form the finitely generated cone
A solution explains how to build from those columns. A certificate instead defines a linear functional . The condition says that its dot product with every generating column is nonnegative, and hence with every point of . But places the target strictly on the other side of the hyperplane . Thus belongs to the dual cone of and rules out every allowed conic combination at once.
A certificate that can be checked by eye. Take the identity matrix
Here is the nonnegative orthant. The equation forces , violating . The certificate gives
The linear functional simply reads the first coordinate: all allowed column combinations have a nonnegative first coordinate, while has first coordinate . For comparison, the target is obtained with .
Figure 1.
Nonnegative column combinations lie in a cone, while a Farkas certificate separates the target
. For , the shaded nonnegative orthant is the conic hull of the two columns. The hyperplane for is the vertical axis; lies strictly on its negative side. The obtainable target illustrates the other alternative. Details.
Why this is useful and striking. A successful search can exhibit one feasible point; an unsuccessful search alone cannot explain why every candidate fails. Farkas' lemma guarantees an equally concrete witness of impossibility for this finite linear problem. Once is supplied, verification needs only matrix multiplication, sign tests and one dot product. The same idea gives a Farkas certificate for linear inequalities and underlies linear programming duality.
Brief geometric proof. The two systems cannot both hold, because they would imply
For existence of a certificate when , closedness of finitely generated cones makes a closed convex cone, so separation from a closed convex cone supplies and for every . Testing each column gives . Concretely, let be the Euclidean projection onto a convex set of onto and set . The projection inequality gives for . Testing and gives , whence . In the example, and . Alternatively, the Fenchel-Moreau theorem applied to the cone's indicator functional proves the same separation through its polar cone.
An original historical source is Julius Farkas's [ref]. The linked archival record verifies the author, title and publication details.
Use the Driven two-level Rabi model in the ordered basis , with Pauli matrices satisfying . Let the Rabi detuning be , in angular frequency, while and are ordinary frequencies in hertz. Assume two identical square pulses, equal drive oscillation phases, constant positive Rabi angular frequency , constant during pulses and flight, and no quantum decoherence or loss. The rotating-wave approximation and isolation of the chosen two-level system are required, just as for the Driven two-level Rabi model. Free flight means no transverse drive; the reference oscillator continues running.
The pulse and free-flight Hamiltonian operators in the rotating frame are
The Pauli matrix multiplication law gives . Thus the exact special unitary group pulse matrix for this effective Hamiltonian operator, and the flight matrix, are
The sequence is . Starting in , matrix multiplication and the Born rule give
This retains the tilted rotation axis and quantum phase accumulated during each finite pulse. At resonance it gives ; with no flight it reduces to the single square-pulse response for duration . Both checks would fail if arbitrary ideal rotations were silently substituted at nonzero Rabi detuning.
For resonant pulse area , and , the central fringe approaches
Its half-height points are , so the central peak's full width at half maximum is in hertz, or in angular frequency. This width assumes an effectively flat pulse envelope across that central peak, one flight time and unit contrast; it is neither a universal finite-pulse width nor a frequency uncertainty. Longer flight lets a smaller frequency difference accumulate the same observable quantum phase.
The two pulses here have the same sign and the plotted outcome is transfer to . The earlier Ramsey interferometry example uses an inverse analysis pulse and reports return to its initial label. Specifying the measured outcome and pulse oscillation phases reconciles the two conventions. In a beam the lengths and speed determine and ; in the Quantronium Ramsey interference experiment electronics set the pulse times for a stationary circuit.
For a finite Markov chain in which an absorbing state is accessible from every state, order the nonabsorbing states first and write
Here is the identity matrix and is the matrix inverse of . With the first hitting time of the absorbing set,
Thus gives the expected occupation count before absorption, including the initial visit; is the absorption probability of state ; and is the expected hitting time of the absorbing set. The vector has every entry one. See Grinstead and Snell, section 11.2, theorems 11.4-11.6.
For the toy routine in absorbing state, direct matrix multiplication gives
The first row of says that starting at gives visits to and to before termination. Their sum is the mean duration . The identities , , and independently verify the inverse, outcome, and duration calculations. The absorbing set must be reachable from every state for these finite formulas: applying them to all nonabsorbing states in the closed-cycle example would leave a persistent cycle in and make singular.
Gaussian elimination with a row swap Created 2026-10-18 Updated 2026-10-20
Consider this original linear system and its augmented matrix:
The first diagonal entry is zero, so it cannot serve as a pivot element. Interchange and to obtain a pivot element of one:
This chooses a convenient nonzero entry for exact arithmetic; partial pivoting would instead interchange and , selecting the largest absolute value, two. For our chosen ordering, the first-column elimination multipliers are and . The elementary row operation gives
The next pivot element is two and the next elimination multiplier is . Applying gives the upper triangular matrix system
Every operation also changed the right-hand side. Back substitution now gives
Checking the original equations, rather than only the reduced ones, verifies the answer by matrix multiplication:
Thus a zero first diagonal entry was a row-order obstruction, not a singular matrix. Indeed, the diagonal product of the final upper triangular matrix is ; the single row interchange makes , since the two row additions preserve the determinant.
Figure 1.
A row interchange supplies the first pivot element; successive elementary row operations create zeros, then back substitution gives
. Details.
Givens rotation Created 2026-09-24 Updated 2026-10-14
A Givens rotation eliminates one component of a vector by rotating just two coordinates, preserving its Euclidean norm. For example,
The length remains five: elimination moves the component into the retained coordinate rather than discarding it. Applied to a matrix, the same operation mixes two rows; QR decomposition by Givens rotations uses these local operations to build an upper triangular matrix.
Figure 1.
A Givens rotation takes clockwise to while preserving the Euclidean norm
. Details.
For indices , the matrix is the identity matrix except for its block on coordinates :
This article uses left matrix multiplication on column vectors. If and , the block is a rotation matrix through angle : the minus sign belongs in the lower left. Its transpose reverses the rotation. Direct multiplication gives and determinant one, so it is an orthogonal matrix; consequently . Only two coordinates change, but this Euclidean norm identity holds for the entire vector.
Givens row-pair update 2026-10-14
Left matrix multiplication by a Givens rotation replaces rows of a matrix by
Both right-hand sides use the old entries. An in-place algorithm must save the first result temporarily, update the second entry, and only then overwrite the first. The reference BLAS DROT implementation follows this order with its temporary DTEMP.[ref]
For elimination in column , construct from . This sets the lower component to zero in exact arithmetic. In floating-point arithmetic the computed expression can leave a small remainder; after computing the other updated entries, explicitly store and . Previously eliminated columns stay zero if both participating rows are already zero there. This is why the ordering in QR decomposition by Givens rotations matters.
For , direct matrix multiplication by the Hadamard gate gives coefficients , proving the displayed identities. This exchanges the two Pauli eigenstates of the Pauli Y gate and agrees with . The phases in the formula are global phases for each separate state; the relative phase inside each state is essential.
Hadamard product 2026-10-06
The matrix with entries . It differs from ordinary matrix multiplication. The Schur product theorem states that it preserves positive semidefiniteness when both factors are positive semidefinite matrices.
Index notation Created 2026-10-20 Updated 2026-10-25
Index notation describes vectors, matrices and tensors by their components relative to a chosen basis. For example, matrix multiplication reads . The free indices identify the resulting component, while is summed. An explicit summation sign avoids depending on an implicit Einstein summation convention.