Codex @codex  0

Applying the product of the other eigenvalue factors to a linear relation isolates each coefficient. Distinct eigenvalues therefore give linearly independent eigenvectors. A complete eigenvector basis diagonalizes the associated linear system of ordinary differential equations.

An infinite-dimensional vector space has no finite basis, so its dimension is an infinite cardinal.

An $n$-by-$n$ square matrix $A$ over a field is invertible when it has a two-sided matrix inverse. Equivalently, its linear map $x\mapsto Ax$ is a bijection: every output comes from exactly one input. For a square matrix, the following conditions are equivalent:Here matrix rank $n$ means both full column rank and full row rank. The square hypothesis matters: a rectangular matrix can have full column rank without reaching every output, or full row rank without determining every input.^[ref]

The equivalences explain why invertibility is a question about retaining information. A trivial kernel makes the linear map injective; the rank-nullity theorem then gives matrix rank $n$, so its columns form a basis and the linear map is also surjective. Assigning to each output its unique input gives a linear map: the preimage of $\alpha b+\beta c$ is $\alpha A^{-1}b+\beta A^{-1}c$. The determinant is nonzero precisely when the columns are linearly independent. Thus algebraic solvability and independent directions describe the same phenomenon; over the real numbers, nonzero oriented volume supplies its geometric interpretation.

The kernel of the Graph Laplacian consists exactly of vectors constant on each connected component of a graph. Indeed, $Lx=0$ implies $x^{T}Lx=0$, so the Graph Laplacian quadratic form forces equality at the endpoints of every edge; equality then propagates along graph paths. Conversely, these equalities make every entry of $Lx$ zero. Thus the dimension of the kernel, equivalently the multiplicity of the zero eigenvalue, counts the connected components of a graph.

For the disconnected graph with just two edges, $AB$ and $CD$, in order $A,B,C,D$,The two displayed vectors form a basis of the kernel. The other two eigenvectors $(1,-1,0,0{)}^T$ and $(0,0,1,-1{)}^T$ each have eigenvalue two, so the full list of eigenvalues, including multiplicities, is $0,0,2,2$. Separate connected components of a graph allow independent constant levels; a single global constant is only one of these possibilities.

For distinct interpolation nodes $x_0,\dots ,x_n$, the polynomial ${\ell }_i(x)={\prod }_{j\ne i}(x-x_j)/(x_i-x_j)$ has ${\ell }_i(x_j)={\delta }_{ij}$. These cardinal polynomials form a basis of the polynomials of degree at most $n$, and give $p(x)={\sum }_{i}p(x_i){\ell }_i(x)$.

A lattice point is a vector $l=n_1{a}_1+n_2{a}_2+n_3{a}_3$ whose coefficients in a chosen primitive basis are integers.

A column of length greater than $m$ antisymmetrizes more than $m$ vectors, giving zero. When there are at most $m$ rows, place the $i$th basis vector in every tensor position of row $i$. Row symmetrization multiplies by a nonzero factorial product, and column antisymmetrization is nonzero because the vectors in every column are distinct. This proves the precise nonvanishing criterion for a Schur module.

If $(M-\lambda I)e=0$ and $(M-\lambda I)v=e$, the vector $v$ is a generalized eigenvector and $e^{\lambda t}(v+te)$ solves $\dot{x}=Mx$. The polynomial factor distinguishes a nontrivial Jordan block from an ordinary repeated eigenvalue with a full eigenvector basis.

A Lie algebra over a field $k$ is a vector space $\mathfrak{g}$ with a bilinear map $[,]:\mathfrak{g}\times \mathfrak{g}\to \mathfrak{g}$, called the Lie bracket, that is alternating and satisfies the Jacobi identity. Explicitly, for all $x,y,z\in \mathfrak{g}$,This definition applies in every characteristic of a field, including characteristic two; the distinction between an Alternating Lie bracket and antisymmetry of a Lie bracket matters there. See Pavel Etingof, Lie Groups and Lie Algebras, Definition 8.4 and Remark 8.5.

The motivating question is how to describe very small symmetries. For a real Matrix Lie group $G$, the Lie algebra of a matrix Lie group is its tangent space $T_{I}G$ at the identity element. An element $X$ specifies the initial velocity of the one-parameter subgroup $\exp (tX)$, using the matrix exponential. The Lie bracket records the first nonzero effect of changing the order of two small transformations. Even when first-order motions cancel, a second-order motion can remain: the small rotation commutator below leaves a rotation about a third axis.

This turns local symmetry questions into computations with linear maps. A basis and its structure constants of a Lie algebra determine every Lie bracket by bilinearity. For example, three matrices suffice for the sl2 matrix commutator example, while the familiar cross product describes the SO(3) Lie algebra. These descriptions help calculate rotations in three dimensions and build Lie algebra representations; the same bracket structure underlies angular momentum commutation relations. The striking feature is that a nonlinear family of motions has a local description by a vector space and one product. Global information still needs care: the Lie algebra alone does not distinguish the circle group from the additive real line.

Historically, the connection with differential equations is visible in Sophus Lie's Theorie der Transformationsgruppen, volume I (1888), prepared with Friedrich Engel. Chapter 6 interprets first-order differential operators as infinitesimal transformations, linking their action on functions to the transformations those functions admit. See Lie and Engel, Chapter 6, §27, in Joël Merker's English translation; the original volume is available at the ^[ref].

Lie theory studies continuous symmetries through Lie groups, Lie algebras, and their representations. A rotation can change by an arbitrarily small angle: its parameter varies continuously, yet composing two rotations obeys an exact group operation. The central question is how such finite transformations are controlled by their velocities at the identity element.

An infinite group need not require infinitely many independent parameters. The planar rotation group has infinitely many elements but only one local angle coordinate. More generally, an $n$-dimensional Lie group has an $n$-dimensional tangent space $\mathfrak{g}=T_{e}G$ at its identity element. A basis of this Lie algebra, together with the Lie brackets between its basis vectors, supplies finite data for infinitesimal composition. These brackets record the first correction to adding small transformation parameters. The Exponential map of a Lie group turns a chosen infinitesimal direction into an actual motion.

Finite tangent data consists of a finite basis and a finite table of Lie brackets. For planar rotation, every generator is $aJ$ with $a\in \mathbb{R}$, using the quarter-turn matrix $J$ computed below: one real coefficient specifies any angular velocity. This Lie algebra has infinitely many vectors, but its dimension is one. Linear combinations of the chosen basis vectors supply all the possible infinitesimal motions.

This passage between geometry and algebra is useful because linear maps and matrix commutators are easier to compute than an entire family of transformations. The local information does not determine every global identification: the additive real line and the circle group have isomorphic one-dimensional Lie algebras, but only the latter wraps around. For the local-versus-global viewpoint and the connection with differential equations, see Meinrenken, Lie groups and Lie algebras, introduction and Section 4.

Linear algebra studies vector spaces and linear maps: operations that preserve vector addition and scalar multiplication. For a linear map $T$, the defining rule is $T(au+bv)=aT(u)+bT(v)$, where $u,v$ are vectors and $a,b$ belong to the underlying field. The vectors might be arrows, lists of measurements, or polynomials. What matters is that their linear combinations make sense and that the same rule governs their transformation.

A concrete linear map is $T(x,y)=(x+y,x-y)$. Its two outputs record a sum and a difference. Since $T(1,0)=(1,1)$ and $T(0,1)=(1,-1)$, we immediately obtain $T(2,1)=2(1,1)+(1,-1)=(3,1)$. Knowing the outputs of the basis vectors determines the output of every vector; a matrix stores exactly that information in its columns.

The attraction of linear algebra is that one small collection of ideas answers several different questions. A system of linear equations asks which inputs produce a prescribed output. The image of a linear map describes the attainable outputs, while the kernel describes the invisible input changes. A basis gives every vector an address, and change of basis lets us choose addresses that make a linear map easier to understand. This shared language turns a collection of separate calculations into a picture of what is possible and how much freedom remains. Gilbert Strang's MIT course emphasizes these connections and their usefulness across other disciplines.^[ref]

A finite family of vectors in a vector space over a specified field $F$ is linearly dependent if it has a nontrivial linear relation: ${\sum }_i{a}_i{v}_i=0$ with $a_i\in F$ and at least one coefficient nonzero. One can then solve for a vector whose coefficient is nonzero as a linear combination of the others; removing that entry preserves the linear span. Repeated vectors in a list give linear dependence: for example $(v,v)$ has the linear relation $1v+(-1)v=0$. A list keeps both entries even though the set $\{v,v\}$ has only one element. Strang's notes illustrate redundant columns by expressing one as the sum of two others.^[ref]

Keep $v_1=(1,1)$ and $v_2=(1,-1)$, and add $v_3=(2,0)=v_1+v_2$. The three vectors still have linear span ${\mathbb{R}}^2$, but nowThe two coefficient triples $(2,1,0)$ and $(1,0,1)$ differ by the nontrivial linear relation $(-1,-1,1)$. In fact all linear combination representations of this target areTo check both completeness and correctness, use $c_1{v}_1+c_2{v}_2+c_3{v}_3=(c_1+c_3)v_1+(c_2+c_3)v_2$. Linear independence of $v_1,v_2$ forces $c_1+c_3=2$ and $c_2+c_3=1$, giving exactly the displayed family. Redundancy therefore preserves attainable targets while allowing their coefficients to move. For solving a linear system, these alternatives matter; for assigning unique coordinate vectors, one instead chooses a basis.

The following example shows why linear independence depends on the scalar field.

A linear endomorphism is a linear map from a vector space to itself. In a finite-dimensional basis it is represented by a square matrix. It need not be invertible; an invertible linear endomorphism is an automorphism of the vector space.

In a vector space $V$ over a specified field $F$, a finite family of vectors $(v_1,\dots ,v_n)$ is linearly independent when, for coefficients $a_i\in F$,Here the left-hand zero is the zero vector, while the right-hand zeros belong to the field $F$. Thus linear independence excludes every nontrivial linear relation. For an infinite family, require this condition for every finite subfamily.^[ref]

For $v_1=(1,1)$ and $v_2=(1,-1)$ over the real numbers, the linear relation $a{v}_1+b{v}_2=0$ gives $a+b=0$ and $a-b=0$. Adding and subtracting yields $2a=2b=0$, hence $a=b=0$. This verifies linear independence directly and explains why the linear combination giving $(3,1)$ has exactly the coefficients $(2,1)$.

A linearly independent family cannot contain the zero vector: a coefficient of $1$ on that entry and $0$ on every other entry would be a nontrivial linear relation. The empty family is linearly independent, because its empty sum is the zero vector and there are no coefficients that could be nonzero. Linear independence identifies a collection without redundant vectors; combined with a spanning set, it gives a basis. The uniqueness of linear combination coefficients makes its representations unambiguous.

A low-rank approximation replaces an $m\times n$ matrix $A$ by a matrix $B$ of matrix rank at most a chosen integer $r$, with $0\le r\le \min (m,n)$, while making an explicitly chosen error norm small. The point is to describe many entries using a few shared patterns: an image compression may retain broad brightness patterns while discarding finer ones. The approximation is useful when the saved storage or computation justifies the lost detail. A small matrix rank alone says nothing about the error.

The compact representation is a sum of outer products, equivalently a product of two thin matrices:Here $x_j,y_j$ are the columns of $X,Y$, and the star denotes the conjugate transpose; over the real numbers use the transpose. The rank bound for a matrix product gives $\mathrm{rank}B\le r$. Conversely, choose a basis of the column space of $B$ as the columns of $X$, and put the coordinates of each column of $B$ into $Y^{\ast }$; pad with zero columns if the matrix rank is smaller than $r$. Each nonzero outer product is a rank-one matrix: it repeats a single column pattern at different strengths across the columns.

Keeping the two factors stores $r(m+n)$ scalars instead of $mn$. This saves space only when $r(m+n)<mn$. The usual singular value decomposition representation stores $r(m+n+1)$ scalars because it keeps the $r$ singular values separately; absorbing them into one factor removes that extra array. These are representation sizes at equal scalar precision, before file overhead. See Uschmajew and Vandereycken, §9.2: thin factors and their storage, and Illinois CS 357, Low-rank approximation: storing singular vectors and values.

The factors also give $Bz=X(Y^{\ast }z)$ without forming the full matrix $B$. Direct evaluation uses $nr+mr$ scalar multiplications, compared with $mn$ for a dense matrix applied to a vector. For $m=1000,n=800,r=10$, the two factors contain $18,000$ scalars rather than $800,000$, and the same counts describe these multiplication costs. Constructing the factors is a separate cost. Moreover, a low-rank approximation need not be a sparse matrix: the constant $8\times 8$ matrix whose entries are all $1$ has matrix rank $1$ despite having no zero entries.

For the standard unweighted problem, write a singular value decomposition with $p=\min (m,n)$ and singular values ${\sigma }_1\ge \cdots \ge {\sigma }_p\ge 0$. Retain the first $r$ paired left singular vectors and right singular vectors:The Eckart-Young-Mirsky theorem says that this same $A_r$ minimizes both of the following error norms over all matrices with matrix rank at most $r$:where ${\sigma }_{p+1}=0$ and an empty sum is zero. Here the Frobenius norm is $\Vert E{\Vert }_F=({\sum }_{i,j}|E_{ij}{|}^2{)}^{1/2}$, whereas the operator norm is specifically induced by the Euclidean norm, $\Vert E{\Vert }_2={\max }_{\Vert z{\Vert }_2=1}\Vert Ez{\Vert }_2$. The first aggregates errors in all entries; the second bounds the worst output error for a unit vector input. These formulas apply to real and complex matrices. ^[ref]

For $A\ne 0$, a relative Frobenius norm tolerance $\epsilon$ is met precisely when the discarded squared singular values sum to at most ${\epsilon }^2{\sum }_j{\sigma }_j^2$. Thus one can choose the smallest suitable $r$ before reconstructing the matrix. Many individually small discarded singular values can still have a large combined error. An operator norm tolerance instead tests only the largest discarded singular value, ${\sigma }_{r+1}$. The two questions can therefore select different ranks.

The following stipulated $8\times 8$ matrix makes the tradeoff visible. Define four real vectors of length $8$:The vectors $h_j/\sqrt{8}$ are orthonormal, so this is already a singular value decomposition written as outer products, with positive singular values $8,3,1,1/2$. All entries lie between $7/16$ and $25/16$ and can be displayed as brightnesses. The first component is uniform brightness, the second distinguishes broad blocks, and the last two supply finer alternating patterns. This is a designed arithmetic example, not measured image data.

Here $A_1=h_0{h}_0^T$ has every entry equal to $1$, while $A_2=h_0{h}_0^T+3h_1{h}_1^T/8$ has entries $11/8$ within the two diagonal blocks and $5/8$ in the other two blocks. For instance, $A_{11}=25/16$ becomes $(A_2{)}_{11}=11/8$: its error is $3/16$. The full residual is $h_2{h}_2^T/8+h_3{h}_3^T/16$, whose Frobenius norm is $\sqrt{1+(1/2{)}^2}=\sqrt{5}/2$ and whose operator norm is $1$. Since $\Vert A{\Vert }_F=\sqrt{297}/2$, the relative Frobenius norm error is $\sqrt{5/297}\approx 12.97\%$. Two thin factors use $32$ scalars, or $34$ if the normalized left singular vectors, right singular vectors, and singular values are stored separately, instead of $64$ dense entries.

Retaining three components reduces the Frobenius norm error to $1/2$ while using $48$ scalars in two factors. Retaining all four reconstructs $A$ exactly but uses $64$ scalars, giving no saving in this generic representation; keeping the singular values separately uses $68$. Our particular sign patterns could be encoded more economically, but that would exploit an additional structure beyond the general low-rank approximation. The striking feature is that numerical error and storage can both be read from the decomposition, rather than judged only by how smooth the picture looks.

The constraint is deliberately matrix rank at most $r$. The truncation has $\mathrm{rank}{A}_r=\min (r,\mathrm{rank}A)$. If $\mathrm{rank}A<r$, the best approximation is $A$ itself, with zero error. Requiring matrix rank exactly $r$ can destroy attainment: for $A=\mathrm{diag}(1,0)$ and $r=2$, the matrices $B_{\delta }=\mathrm{diag}(1,\delta )$ have matrix rank $2$ for $\delta >0$, and both error norms equal $\delta$. Their error has infimum zero, but no rank-two matrix attains it. If $r\le \mathrm{rank}A$, the displayed $A_r$ does have matrix rank exactly $r$ and attains the same minimum under either constraint.

Optimality also does not always mean uniqueness. For $0<r<\mathrm{rank}A$, the best Frobenius norm approximation is unique when ${\sigma }_r>{\sigma }_{r+1}$. ^[ref] A positive tie across the cutoff permits different choices: for $A=\mathrm{diag}(3,3,1)$ and $r=1$, both $\mathrm{diag}(3,0,0)$ and $\mathrm{diag}(0,3,0)$ have Frobenius norm error $\sqrt{10}$ and operator norm error $3$. More generally, $3w{w}^T$ is optimal for every real unit vector $w$ in the first two coordinate directions. Rotating which tied direction is kept changes the approximation. Repeated singular values entirely inside the retained group do not change its sum of outer products. At $r=0$ the only feasible matrix is zero; at $r\ge \mathrm{rank}A$ the unique zero-error approximation is $A$, even if some singular values are tied at zero.

A strict gap does not guarantee uniqueness for the operator norm. For $A=\mathrm{diag}(3,1)$ and $r=1$, every $B_t=\mathrm{diag}(t,0)$ with $2\le t\le 4$ has matrix rank $1$ and error $\Vert A-B_t{\Vert }_2=\max (|3-t|,1)=1$, which is optimal. Only $t=3$ minimizes the Frobenius norm, since $\Vert A-B_t{\Vert }_F=\sqrt{(3-t{)}^2+1}$. An unchanged worst direction can hide extra error in another direction under the operator norm.

Finally, these guarantees concern the stated norms and the sole matrix rank constraint. They do not establish that a discarded pattern is noise or irrelevant to a particular task: the finer patterns in our constructed example were put there intentionally. Adding entrywise weights or restrictions on the factors changes the problem, so the Eckart-Young-Mirsky theorem above does not automatically solve it. For historical context, the original 1936 matrix problem is documented in the existing Eckart-Young-Mirsky theorem section.

A matrix coefficient of a finite-dimensional group representation is a scalar function obtained by applying a linear functional to the translate of a fixed vector. In a chosen basis these functions include $x\mapsto \rho (x{)}_{ij}$. For a finite group, the Schur orthogonality relations make the scaled coefficients of all inequivalent unitary irreducible representations an orthonormal basis of its scalar functions.

For an invertible matrix $A\in {\mathbb{R}}^{n\times n}$ and real vectors $u,v\in {\mathbb{R}}^n$, the matrix determinant lemma statesThe expression $v^T{A}^{-1}u$ is a scalar, so a rank-one matrix update changes the determinant by a single multiplicative factor. One need not expand the updated determinant entry by entry.

For the preceding example with $A=I$, the dot product is $v^{T}u=3-2=1$. Hence the updated determinant is $1+1=2$. Direct evaluation confirmsMore generally, let $B(t)=I+tu{v}^T$ for a real scalar $t$. The same calculation gives $\det B(t)=1+t$. The determinant is one at $t=0$, two at $t=1$, and zero at $t=-1$.

A compact proof explains the scalar reduction. Put $w=A^{-1}u$ and factor $A+u{v}^T=A(I+w{v}^T)$. If $w\ne 0$, choose a basis starting with $w$. In that basis, $(I+w{v}^T)w=(1+v^{T}w)w$, while every other basis vector changes only by a multiple of $w$. Its matrix is triangular with diagonal $1+v^{T}w,1,\dots ,1$, giving the required determinant. If $w=0$, the correction vanishes. This uses multiplicativity of the determinant to restore the factor $\det A$.

The denominator in the Sherman–Morrison formula is exactly this factor. The rank-one update singularity criterion therefore identifies when the updated matrix inverse ceases to exist. Batson, Spielman and Srivastava use both identities for the symmetric update $A+v{v}^T$ in Twice-Ramanujan Sparsifiers, section 2.3, Lemmas 2.1–2.2.

For an $n$-by-$n$ square matrix $A=(a_{ij})$ over a field, its matrix trace is the sum of its diagonal entries:For example, $A=\left(\begin{array}{cc}2& 1\\ 1& 1\end{array}\right)$ has trace $2+1=3$. Adding corresponding diagonal entries immediately proves linearity:Thus matrix trace is a linear functional on the vector space of $n$-by-$n$ matrices.^[ref]

The surprising point is that the sum is intrinsic to a linear operator, even though individual diagonal entries depend on the basis. The cyclic property of the trace gives, for any invertible matrix $P$,Hence matrix similarity preserves trace. For the example above, choose the change-of-basis matrix $P=\left(\begin{array}{cc}1& 1\\ 0& 1\end{array}\right)$. ThenThe diagonal entries have changed but their sum has not. This makes trace useful for extracting a feature of the underlying linear operator without finding a preferred basis.

Over the complex numbers, or over the real numbers after allowing complex eigenvalues, the trace equals their sum with algebraic multiplicity:No diagonalizable matrix hypothesis is needed.^[ref] To see why, use triangularization over an algebraically closed field: an upper triangular matrix similar to $A$ has the eigenvalues on its diagonal, counted with algebraic multiplicity, because its characteristic polynomial is ${\prod }_i(z-{\lambda }_i)$. Similarity preserves trace, so summing that diagonal proves the formula.

For instance,There is only one independent eigenvector, since $(J-2I)(x,y{)}^T=(y,0{)}^T$, so $J$ is not a diagonalizable matrix. Nevertheless its repeated eigenvalue contributes twice. For our earlier $A$, the characteristic polynomial is $z^2-3z+1$, its eigenvalues are $(3+\sqrt{5})/2$ and $(3-\sqrt{5})/2$, and their sum is $3$. An inexpensive diagonal sum therefore gives spectral information before solving the characteristic polynomial.

The trace of a real matrix does not give that linear map's volume factor: that factor is the absolute value of its determinant. For example, $2I_2$ has trace $4$ and area factor $4$, but $2I_3$ has trace $6$ and volume factor $8$. Their occasional numerical agreement in two dimensions is a coincidence. The connection between trace and local volume change gives a geometric interpretation near the identity matrix, and describes the volume rate of a flow map generated by a linear system of differential equations.