Active planar rotation 2026-10-14
For column coordinate vectors in the fixed Cartesian basis, an active planar rotation sends to . For radians,
The endpoint travels counterclockwise through along a circle of radius . Its Euclidean norm stays . The second Cartesian basis direction becomes , whose dot product with is still zero.
Figure 1.
An active counterclockwise planar rotation by : the vector moves while the Cartesian basis stays fixed
. Details.
A quick sign check is to look at the first column: must point above the horizontal axis for . The columns have unit Euclidean norm and zero dot product, so ; the determinant is . Finally, the transpose negates the off-diagonal entries, giving : turning back reverses the action. This is the same matrix used for passive rotation of coordinates, though that interpretation keeps the vector fixed.
Adjoint in weighted coordinates Created 2026-10-14 Updated 2026-10-31
Suppose a linear map between finite-dimensional Hilbert spaces has matrix , and their inner products in the chosen bases are and . The Gram matrices are positive-definite matrices. Writing for the ordinary conjugate transpose, the coordinate matrix of the adjoint operator is
Indeed, the defining inner product identity says for all . Thus simply transposing and conjugating is valid in orthonormal bases, where both Gram matrices are the identity matrix.
For the weighted inner product and the coordinate swap , the adjoint operator has matrix . With and , both inner products equal . The coordinate swap is its own ordinary transpose, but is not self-adjoint in this weighted inner product.
Backward error of an approximate eigenpair Created 2026-09-24 Updated 2026-10-25
If , the smallest operator-norm perturbation that makes an exact eigenpair has norm . One such rank-one perturbation is
Bilinear form Created 2026-09-24 Updated 2026-10-14
A bilinear form takes two vectors in a vector space over a field and returns a scalar in . Its defining property is separate linearity: holding either input fixed gives a linear functional of the other. For scalars and vectors ,
This is a scalar-valued bilinear map; the chosen field is part of the definition.[ref]
Separate linearity does not mean that is a linear map on the vector space with componentwise vector addition and scalar multiplication. Over the real numbers, , whereas a linear map on the pair would give . The simplest example is on the one-dimensional vector space : fixing gives , but doubling both and quadruples their product.
A bilinear form measures how two inputs interact. The dot product measures alignment; a symmetric bilinear form can describe a quadratic form; an alternating bilinear form can measure signed area. A nondegenerate bilinear form turns each vector into a unique linear functional and so connects geometry with the dual space. The same short definition therefore supports lengths, bilinear orthogonal complements, and symplectic vector spaces, depending on the additional properties imposed.
Over the complex numbers, a bilinear form is linear in both inputs without conjugating either scalar. A Hermitian form instead is a sesquilinear form and conjugates when its inputs are exchanged. For complex matrices, the adjoint used with Hermitian forms and matrix C-star algebras is the conjugate transpose, whereas a bilinear form uses the ordinary transpose . For example, has , while the Hermitian form has .[ref]
Rotate the standard basis through using the rotation matrix whose columns are the new basis vectors:
Here and , so is an orthogonal matrix with , where denotes transpose and is the identity matrix. For the same vector , its coordinate vector in this Cartesian basis is
Thus and : travel forwards along and a little backwards along to reconstruct . The orthogonal projections onto the two axes give these signed coordinates because the measuring vectors are both perpendicular and of unit length.
Figure 1.
The vector reconstructed from its orthogonal projections onto a rotated Cartesian basis
. Details.
The squared norm remains
This is useful when choosing axes aligned with a geometric feature: the coordinates change but distances and dot products retain the same sum-of-products formula. The two directions of the change of basis must be kept straight: reconstructs the vector in old coordinates, , and extracts its new coordinates. Applying to the old coordinate vector instead would actively rotate the vector.
A short proof explains both formulas. If , multiplication by gives . Then . Perpendicular unit-length columns make the matrix inverse equal the transpose; this is the computational convenience supplied by a Cartesian basis.
Cholesky decomposition Created 2026-09-28 Updated 2026-10-25
Every real symmetric matrix that is a positive-definite matrix has a unique Cholesky decomposition , where is a lower triangular matrix with positive diagonal entries.[ref] For a complex Hermitian positive-definite matrix, the corresponding factorization is , with the conjugate transpose and the diagonal of positive and real.[ref] The factorization turns a linear system into two triangular systems.
For example, matching entries in gives
Choose the positive square roots in order: , and . Therefore
The second radicand, , is the scalar Schur complement remaining after eliminating the first variable. A positive diagonal of alone would not guarantee that this quantity is positive; Sylvester's criterion explains the relevant leading principal minors.
Figure 1. . Original exact two-by-two example. Multiplying the displayed lower triangular matrix by its transpose recovers ; forward substitution and backward substitution in a triangular system give for . Details.
For that right-hand side, first use forward substitution to solve : and , so . Then use backward substitution in a triangular system to solve : and , giving . No matrix inverse is required. For several right-hand sides with the same , keep the same factor and repeat only these triangular solves.
André-Louis Cholesky developed his calculation in the setting of geodetic adjustment: measured angles and lengths must satisfy geometric relations, while corrections should be small. The manuscript reproduced in Claude Brezinski's study is titled Sur la résolution numérique des systèmes d’équations linéaires and ends with the date 2 December 1910 at Vincennes. The manuscript reports a 56-equation calculation adjusting altitudes in an Algerian triangulation network. Its date establishes when this written account existed, rather than an exact invention date.[ref]
A related algebraic reason for the method's usefulness is that normal equations involve a Gram matrix. If a real matrix has linearly independent columns, then
so is a positive-definite matrix. That gives the structure needed by a Cholesky decomposition. The historical adjustment problem instead produces from independent constraint rows; the same argument applies to its transpose. In numerical least squares, explicitly forming normal equations squares the condition number of a full-column-rank design matrix; a QR decomposition can therefore be preferable for accuracy. Indeed, the extreme singular values give . The factorization remains useful when a positive-definite matrix is already the natural input.
Cholesky decomposition as a sum of squares Created 2026-10-18 Updated 2026-10-31
For any real vector , a Cholesky decomposition gives the quadratic form identity
Since is an invertible matrix, this squared Euclidean norm is positive whenever . The factorization thus certifies positive definiteness and gives coordinates in which the quadratic form is a sum of ordinary squares. For the matrix above,
The level set is an ellipse; the linear map sends it to the unit circle. The points and go to and , respectively.
Figure 1. . Original exact ellipse and its image under . Matching point colors show two mapped points; the axes use their own coordinates. This transformation need not be a rotation or a uniform scaling. Details.
The same factor works in the reverse direction for Gaussian random vectors. If has independent unit-variance, zero-mean Gaussian random variables as coordinates, has covariance matrix . In this example and : their variances are and , while their covariance is . This is a concrete way to generate specified correlations from independent draws.
A triangular factor is not generally a square root of a matrix in the sense . Here . The Cholesky decomposition uses the product with the transpose, which is also what makes the quadratic form a squared Euclidean norm.
For a matrix over the complex numbers with the standard Hermitian inner product, the linear system is solvable exactly when
The star is the conjugate transpose; for matrices over the real numbers, use the transpose. Necessity follows from . For sufficiency, , and taking the orthogonal complement again recovers in finite dimension. This is the cokernel obstruction expressed as scalar equations. [ref]
For , , so the single compatibility equation is . If it holds, the kernel supplies the remaining input freedom; if it fails, changing the input cannot remove the obstruction.
A matrix and its transpose have the same characteristic polynomial, allowing two independent Gershgorin eigenvalue counting arguments for one spectrum. If isolated row discs certify simple eigenvalues near one group of centres and isolated column discs certify the remaining ones, pairwise separation of all the certified small discs establishes a simple spectrum even if neither full family has pairwise disjoint discs.
For example, let , the diagonal centres be , and all adjacent off-diagonal entries have magnitude strictly below . Add a corner entry . The row radii are below ; the column radii reverse the two endpoint bounds. If , rows isolate centres and columns isolate centres . The nearest potentially enlarged-disc contact has centre separation , while the radius sum is strictly less than . Each of the certified small discs contains one simple eigenvalue. For real entries these eigenvalues are real, since a nonreal conjugate pair would put two roots in the same conjugation-invariant isolated disc.
This is a sufficient uniform certificate. Overlap beyond this bound does not prove repeated eigenvalues; individual matrices can have simple real spectra well beyond a Gershgorin disc separation bound.
Controllability Gramian 2026-10-14
For and the constant matrices over the real numbers in the Kalman rank criterion, the finite-time Controllability Gramian accumulates the influence of all command times. Its column space equals that of the controllability matrix, and is a symmetric positive-definite matrix exactly when the controllability matrix has rank of a matrix . If so, put . Among square-integrable functions steering to at , the command
minimizes , with value . Here in a matrix superscript denotes transpose. This cost is a squared input norm, not necessarily physical energy, and the optimizing command need not satisfy an amplitude bound. [ref]
For the double integrator, , so direct integration gives
For a rest-to-rest distance , and the minimum squared input norm is . Compare the two constant commands in the double integrator example, whose cost is : both attain the target, but the smooth command costs less. At , , however, peaks at , so it violates while the two constant commands remain admissible. Controllability, optimal control, and constrained reachability in control theory answer different questions.
Dual lattice Created 2026-09-28 Updated 2026-10-14
The dual lattice of a full-rank Euclidean lattice is
Here is the Euclidean inner product. If the columns of an invertible matrix form a basis of , then
Indeed, belongs to the dual lattice exactly when ; denotes the transpose of the matrix inverse. The reciprocal covolume follows by taking the determinant. With the Fourier transform kernel , these are precisely the frequencies of the characters of a real torus: translating by any leaves unchanged. In one dimension, for : increasing the spacing in physical space decreases the spacing in frequency space.
Fredholm alternative for a matrix Created 2026-09-24 Updated 2026-10-03
The system is solvable exactly when is orthogonal to every vector in .
Gershgorin disc Created 2026-10-06 Updated 2026-10-31
A row Gershgorin disc of a square matrix is the closed disk in the complex plane with centre and radius . A zero radius gives a single point. The Gershgorin disc theorem places every eigenvalue in the union of these disks. For a Hermitian matrix, the real intervals cut out by the Gershgorin discs bound the eigenvalues, including energy levels when the matrix represents a Hamiltonian operator.
Applying the Gershgorin circle theorem to the transpose gives column Gershgorin discs with the same centres and radii . Since and have the same characteristic polynomial, every eigenvalue lies in the intersection of the row-disc union and the column-disc union. This is an intersection of two unions; one cannot generally replace each row radius by . An isolated row Gershgorin disc contains exactly one eigenvalue by Gershgorin eigenvalue counting.[ref]
Givens rotation Created 2026-09-24 Updated 2026-10-14
A Givens rotation eliminates one component of a vector by rotating just two coordinates, preserving its Euclidean norm. For example,
The length remains five: elimination moves the component into the retained coordinate rather than discarding it. Applied to a matrix, the same operation mixes two rows; QR decomposition by Givens rotations uses these local operations to build an upper triangular matrix.
Figure 1.
A Givens rotation takes clockwise to while preserving the Euclidean norm
. Details.
For indices , the matrix is the identity matrix except for its block on coordinates :
This article uses left matrix multiplication on column vectors. If and , the block is a rotation matrix through angle : the minus sign belongs in the lower left. Its transpose reverses the rotation. Direct multiplication gives and determinant one, so it is an orthogonal matrix; consequently . Only two coordinates change, but this Euclidean norm identity holds for the entire vector.
Hermitian conjugation Created 2026-09-28 Updated 2026-10-14
Hermitian conjugation is the operation of taking the adjoint operator, written . For a matrix in orthonormal bases, it takes the complex conjugate of each entry and exchanges rows with columns:
For a real matrix it is the ordinary transpose. For the complex matrix in the adjoint operator example, its second row becomes the second column of . This is the same conjugate transpose used in MIT's treatment of complex inner products and Hermitian matrices.[ref]
For bounded operators with matching source and target Hilbert spaces, Hermitian conjugation satisfies
The reversal of function composition is forced by moving both linear operators across an inner product:
The calculation identifies the adjoint operator by the uniqueness theorem in Melrose's MIT notes.[ref]
For a computed order check, let and reuse . Then
Keeping the original order gives a different matrix. Likewise : taking the adjoint operator is conjugate-linear as an operation on linear operators, although each individual is a linear map on vectors.
Kalman prediction step 2026-10-14
The Kalman prediction step advances the previous posterior mean and posterior covariance matrix through the state transition:
The minus superscript marks information before observing . The transpose and matrix multiplication propagate the old uncertainty; adds the uncertainty of the new process noise. A known control input changes the mean but contributes no additional variance. If the control input itself is uncertain, that uncertainty needs its own model.
Laplace expansion Created 2026-10-18 Updated 2026-10-20
Expand the determinant along any fixed row using the displayed formula, where deletes row and column . The signed matrix minors are its cofactors. Applying the same formula to the transpose gives the column expansion. The row formula follows from the Leibniz formula for determinants by grouping permutations according to the selected column in row . For a lower triangular matrix, expanding along the first row reduces the determinant to times a smaller triangular determinant; induction gives the product of all diagonal entries, even when some vanish.
Low-rank approximation 2026-10-14
A low-rank approximation replaces an matrix by a matrix of matrix rank at most a chosen integer , with , while making an explicitly chosen error norm small. The point is to describe many entries using a few shared patterns: an image compression may retain broad brightness patterns while discarding finer ones. The approximation is useful when the saved storage or computation justifies the lost detail. A small matrix rank alone says nothing about the error.
The compact representation is a sum of outer products, equivalently a product of two thin matrices:
Here are the columns of , and the star denotes the conjugate transpose; over the real numbers use the transpose. The rank bound for a matrix product gives . Conversely, choose a basis of the column space of as the columns of , and put the coordinates of each column of into ; pad with zero columns if the matrix rank is smaller than . Each nonzero outer product is a rank-one matrix: it repeats a single column pattern at different strengths across the columns.
Keeping the two factors stores scalars instead of . This saves space only when . The usual singular value decomposition representation stores scalars because it keeps the singular values separately; absorbing them into one factor removes that extra array. These are representation sizes at equal scalar precision, before file overhead. See Uschmajew and Vandereycken, §9.2: thin factors and their storage, and Illinois CS 357, Low-rank approximation: storing singular vectors and values.
The factors also give without forming the full matrix . Direct evaluation uses scalar multiplications, compared with for a dense matrix applied to a vector. For , the two factors contain scalars rather than , and the same counts describe these multiplication costs. Constructing the factors is a separate cost. Moreover, a low-rank approximation need not be a sparse matrix: the constant matrix whose entries are all has matrix rank despite having no zero entries.
For the standard unweighted problem, write a singular value decomposition with and singular values . Retain the first paired left singular vectors and right singular vectors:
The Eckart-Young-Mirsky theorem says that this same minimizes both of the following error norms over all matrices with matrix rank at most :
where and an empty sum is zero. Here the Frobenius norm is , whereas the operator norm is specifically induced by the Euclidean norm, . The first aggregates errors in all entries; the second bounds the worst output error for a unit vector input. These formulas apply to real and complex matrices. [ref]
For , a relative Frobenius norm tolerance is met precisely when the discarded squared singular values sum to at most . Thus one can choose the smallest suitable before reconstructing the matrix. Many individually small discarded singular values can still have a large combined error. An operator norm tolerance instead tests only the largest discarded singular value, . The two questions can therefore select different ranks.
The following stipulated matrix makes the tradeoff visible. Define four real vectors of length :
The vectors are orthonormal, so this is already a singular value decomposition written as outer products, with positive singular values . All entries lie between and and can be displayed as brightnesses. The first component is uniform brightness, the second distinguishes broad blocks, and the last two supply finer alternating patterns. This is a designed arithmetic example, not measured image data.
Figure 1.
Shared patterns in a stipulated matrix
. The first three panels use the same brightness scale. Rank one retains uniform brightness; rank two adds the broad blocks. The signed residual has its own color scale to expose the discarded patterns. The factors apply a linear map through just intermediate coordinates. Details.
Here has every entry equal to , while has entries within the two diagonal blocks and in the other two blocks. For instance, becomes : its error is . The full residual is , whose Frobenius norm is and whose operator norm is . Since , the relative Frobenius norm error is . Two thin factors use scalars, or if the normalized left singular vectors, right singular vectors, and singular values are stored separately, instead of dense entries.
Figure 2.
Error falls as storage rises for the stipulated matrix
. The exact Frobenius norm and operator norm errors are plotted for . The storage panel counts the two thin factors, or the normalized left singular vectors and right singular vectors plus separate singular values, at equal scalar precision. Lines join discrete rank choices; they do not represent intermediate ranks or measured data. Details.
Retaining three components reduces the Frobenius norm error to while using scalars in two factors. Retaining all four reconstructs exactly but uses scalars, giving no saving in this generic representation; keeping the singular values separately uses . Our particular sign patterns could be encoded more economically, but that would exploit an additional structure beyond the general low-rank approximation. The striking feature is that numerical error and storage can both be read from the decomposition, rather than judged only by how smooth the picture looks.
The constraint is deliberately matrix rank at most . The truncation has . If , the best approximation is itself, with zero error. Requiring matrix rank exactly can destroy attainment: for and , the matrices have matrix rank for , and both error norms equal . Their error has infimum zero, but no rank-two matrix attains it. If , the displayed does have matrix rank exactly and attains the same minimum under either constraint.
Optimality also does not always mean uniqueness. For , the best Frobenius norm approximation is unique when . [ref] A positive tie across the cutoff permits different choices: for and , both and have Frobenius norm error and operator norm error . More generally, is optimal for every real unit vector in the first two coordinate directions. Rotating which tied direction is kept changes the approximation. Repeated singular values entirely inside the retained group do not change its sum of outer products. At the only feasible matrix is zero; at the unique zero-error approximation is , even if some singular values are tied at zero.
A strict gap does not guarantee uniqueness for the operator norm. For and , every with has matrix rank and error , which is optimal. Only minimizes the Frobenius norm, since . An unchanged worst direction can hide extra error in another direction under the operator norm.
Finally, these guarantees concern the stated norms and the sole matrix rank constraint. They do not establish that a discarded pattern is noise or irrelevant to a particular task: the finer patterns in our constructed example were put there intentionally. Adding entrywise weights or restrictions on the factors changes the problem, so the Eckart-Young-Mirsky theorem above does not automatically solve it. For historical context, the original 1936 matrix problem is documented in the existing Eckart-Young-Mirsky theorem section.
Matrix proportional to its transpose Created 2026-10-20 Updated 2026-10-25
For a matrix over the real numbers or complex numbers, applying the transpose twice to gives . A nonzero matrix is therefore possible only for or . These cases are precisely the symmetric matrices and skew-symmetric matrices, respectively. For every other scalar, the only solution is the zero matrix.