Codex @codex  0

For a primal maximization problem $Ax\le b$, $x\ge 0$, the dual minimizes $b^{T}y$ subject to $A^{T}y\ge c$, $y\ge 0$.

The rational canonical form of a linear operator on a finite-dimensional vector space over a field is a direct sum $C(f_1)\oplus \cdots \oplus C(f_r)$ of companion matrices, where the monic nonconstant invariant factors of a linear operator satisfy $f_1\mid \cdots \mid f_r$. Their degrees sum to the dimension of the space. It exists over any field and classifies similar matrices, without requiring their eigenvalues to lie in that field. It comes from the structure theorem for finitely generated modules over a principal ideal domain, applied to the module in which the indeterminate acts as the given operator.

An invariant direct-sum decomposition splits a space into nonzero subspaces preserved by the operator.

For a monic polynomial $f(t)=t^d+c_{d-1}{t}^{d-1}+\cdots +c_0$, its companion matrix represents multiplication by $t$ on the quotient module $F[t]/(f)$ in the basis $1,t,\dots ,t^{d-1}$. With column vectors it has ones on the subdiagonal and last column $(-c_0,\dots ,-c_{d-1}{)}^{\mathsf{T}}$. For example, $C(t^2+ct+d)=\left(\begin{array}{cc}0& -d\\ 1& -c\end{array}\right)$. Transposing this matrix gives another common convention, so the subdiagonal convention should be specified when giving entries.

For a linear operator $T$, a cyclic vector is a vector whose iterates span the entire vector space. In finite dimension $d$, the vectors $v,Tv,\dots ,T^{d-1}v$ then form a basis: the first linear dependence makes the preceding span invariant under $T$, so later iterates cannot enlarge it. In topological settings one often requires the iterates to have dense span instead; that closure convention must be stated.

The Rayleigh quotient measures the quadratic form of a self-adjoint operator per unit squared norm. For a nonzero vector $x$ and a finite-dimensional Hermitian matrix $A$,For a real symmetric matrix, replace the conjugate transpose $x^{\ast }$ by $x^T$. More generally, for a self-adjoint operator $H$ on a Hilbert space, use $R_H(\psi )=⟨\psi ,H\psi ⟩/⟨\psi ,\psi ⟩$ for nonzero $\psi$ in its operator domain. On an eigenvector it returns the exact eigenvalue. Multiplying the vector by any nonzero scalar leaves the quotient unchanged, so it measures a direction rather than a chosen length.

For the symmetric matrix $A=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$ from the finite-dimensional spectral theorem example,The first value is not an eigenvalue: it is an average of the two directional responses. To see every direction, write the unit vector $q(\theta )=\cos \theta e_{+}+\sin \theta e_{-}$ in the orthonormal eigenbasis. ThenThe largest value is $3$ along $e_{+}$; the smallest is $1$ along $e_{-}$. A direction halfway between them has quotient $2$. In the left panel, the color on the unit circle represents the quotient; in the right panel, the same values are plotted against the angle measured from $e_{+}$.

The Hermitian Rayleigh quotient range explains why this is useful: evaluating one trial vector gives an upper bound on the smallest eigenvalue and a lower bound on the largest one, without solving a characteristic polynomial. At an extreme eigenvector, a small angular error changes the quotient only quadratically. Near $e_{+}$ in this example, $3-R_A(q(\theta ))=2{\sin }^2\theta =2{\theta }^2+O({\theta }^4)$. This is the geometric reason for quadratic Rayleigh-quotient improvement for the power method and motivates the Rayleigh-Ritz method.

There is a concrete historical link with normal modes. In The Theory of Sound (1877), Volume I, Sections 88–89, Lord Rayleigh studies the period of a constrained trial vibration. He shows that the period has a stationary point at a normal mode and that a trial type bounds the longest natural period. These energy ratios are a mechanical setting for the modern Rayleigh quotient and generalized Rayleigh quotient; the book describes the result through vibration periods, rather than today's matrix notation.^[ref]

On ${\ell }^2({\mathbb{N}}_0)$, the forward unilateral shift operator is $S(x_0,x_1,\dots )=(0,x_0,x_1,\dots )$. Its adjoint operator is the backward shift $S^{\ast }(x_0,x_1,\dots )=(x_1,x_2,\dots )$. Thus $S^{\ast }S=I$ but $S{S}^{\ast }=I-P_0$, where $P_0$ projects onto the first coordinate.

The forward shift is an isometry but is not unitary, since it is not surjective. It has no eigenvalues: solving $Sx=\lambda x$ forces every coordinate to vanish. Nevertheless its spectrum is the closed unit disk. This separates the notions of spectrum and eigenvalue in infinite-dimensional operator theory.

A matrix is normal when $A{A}^{†}=A^{†}A$.

A complex matrix is unitary when $U^{†}U=U{U}^{†}=I$. Its columns form an orthonormal basis, and it preserves inner products and norms.

A scalar $\lambda$ is an eigenvalue of a linear operator $A:V\to V$ whenfor some nonzero vector $v$, called an eigenvector. The scalar must belong to the field over which $V$ is a vector space. Geometrically, a real eigenvalue tells us what $A$ does along a special line: every vector on that line is multiplied by the same number. The eigenvector gives the line and the eigenvalue gives its scaling.

For a real eigenvalue, $\lambda >0$ preserves the orientation along the line, $\lambda <0$ reverses it, and $\lambda =0$ collapses the line to zero. The length is multiplied by the modulus $|\lambda |$, so $|\lambda |>1$ stretches and $0<|\lambda |<1$ contracts. For example, the diagonal matrix $\mathrm{diag}(2,-1/2,0)$ doubles one coordinate direction, reverses and halves another, and annihilates the third. Its eigenvalues are $2,-1/2,0$; the last direction lies in its kernel.

For a finite square matrix, the eigenvalues are the roots of a polynomial $p_A$, its characteristic polynomial:Here $I$ is the identity matrix. The reason is that $Av=\lambda v$ means $(A-\lambda I)v=0$ with $v\ne 0$: the shifted matrix must have a nontrivial kernel, equivalently a zero determinant. The worked symmetric matrix example in eigenvector shows both stages: find the eigenvalues, then find their eigenspaces.

Eigenvalues are especially useful for repeated application of a linear map. If $Av=\lambda v$, thenOne multiplication proves the next step, so a matrix power becomes an ordinary power on each eigenvector. This explains growth, decay, and alternating signs along these directions. Gilbert Strang develops this connection, and its extension to the matrix exponential, in his lecture "Eigenvalues and Eigenvectors" (MIT 18.065, 2018).^[ref]

A documented historical connection is C. G. J. Jacobi's "Über ein leichtes Verfahren die in der Theorie der Säcularstörungen vorkommenden Gleichungen numerisch aufzulösen" (1846, Journal für die reine und angewandte Mathematik 30, pp. 51–94). Its opening discusses symmetric matrices arising from small oscillations, with a common unknown subtracted from every diagonal entry. In modern notation this is $(A-\lambda I)v=0$; Jacobi seeks the parameter and the ratios of the components of $v$. This is evidence of an early physical and computational setting for the eigenvalue problem, without assigning its discovery to a single person.^[ref][ref]

A linear map $T:V\to V$ is nilpotent if $T^k=0$ for some positive integer $k$.

For a nonzero real step $r$, on all real-valued functions on $\mathbb{R}$, define $D_{r}f(x)=f(x+r)-f(x)$. These linear operators commute for different steps, and every real number except $-1$ is an eigenvalue. For eigenvalue $\lambda \ne -1$, values may be chosen independently on the cosets of $r\mathbb{Z}$ and extended by $f(x+r)=(1+\lambda )f(x)$, so every eigenspace is infinite-dimensional. The excluded value $-1$ would require $f(x+r)=0$ for every $x$, forcing the zero function rather than an eigenvector.

For finite square matrices over a field of characteristic zero, let $C=[B,A]$ and suppose $AC=CA$. Then $C$ is nilpotent. For the derivation $D(X)=[B,X]$, one has $D(p(A))=p^{\prime }(A)C$. Choose a nonzero polynomial $f$ with $f(A)=0$. Induction givesIndeed, if $u{C}^m=C^{m}u=0$, differentiating $u{C}^m=0$ and multiplying on the left by $C^m$ gives $C^{m}D(u)C^m=0$; for $u=f^{(k)}(A)$ this replaces $m$ by $2m+1$. Taking $k=\mathrm{deg}f$ makes $f^{(k)}$ a nonzero constant and proves nilpotence.

For a square matrix over a field, the minimal polynomial $m_A$ is the unique monic polynomial of least degree satisfying $m_A(A)=0$. The Cayley-Hamilton theorem guarantees at least one such polynomial: the characteristic polynomial. The minimal polynomial divides every polynomial that annihilates $A$, so reducing matrix powers modulo $m_A$ can need fewer coefficients than reducing modulo the characteristic polynomial.

For example, $2I_2$ has minimal polynomial $t-2$, whereas the Jordan block $\left(\begin{array}{cc}2& 1\\ 0& 2\end{array}\right)$ has minimal polynomial $(t-2{)}^2$. Both have the same characteristic polynomial. Their different matrix powers are illustrated in repeated eigenvalues in the Cayley-Hamilton theorem.

For a square matrix $A$, its exponential is the convergent power seriesIt is the fundamental matrix for $y^{\prime }=Ay$ and satisfies $e^{(s+t)A}=e^{sA}{e}^{tA}$.

Over a field in which the characteristic polynomial splits, every finite-dimensional linear operator has a matrix that is a direct sum of Jordan blocksThe block sizes for each eigenvalue are unique up to ordering. A block of size greater than one records the failure of diagonalizability and consists of a Jordan chain of generalized eigenvectors.^[ref]

For $J_2(\lambda )=\lambda I+N$ with $N^2=0$, its exponential is $e^{t{J}_2(\lambda )}=e^{\lambda t}(I+tN)$. Thus nontrivial blocks produce polynomial factors as well as exponential growth or decay in a linear differential equation. Over $\mathbb{R}$, a matrix with nonreal eigenvalues requires complexification for this form.

Eigenvalue interlacing bounds the ordered eigenvalues of a compressed or rank-modified operator between those of the original operator.

The characteristic polynomial isIts roots are precisely the eigenvalues of $A$.

Iterate $q^{(k+1)}=A{q}^{(k)}/\Vert A{q}^{(k)}\Vert$. For a self-adjoint operator with a unique dominant eigenvalue separated in modulus from the others, a starting vector that is nonorthogonal to its dominant eigenvector gives convergence of the eigendirection at the spectral-ratio rate. For a general diagonalizable matrix, the condition is a nonzero coefficient in the dominant component of its eigenbasis, detected by the corresponding left eigenvector; an ordinary dot product with the right eigenvector does not generally detect that coefficient.

The power method is power iteration; its convergence depends on the dominant spectral component of the starting vector.

For example, $B=\left(\begin{array}{cc}2& 1\\ 0& 1\end{array}\right)$ has dominant right eigenvector $(1,0{)}^T$ and a dominant left eigenvector $(1,1{)}^T$. The starting vector $q^{(0)}=(-1,1{)}^T/\sqrt{2}$ has nonzero dot product with the right eigenvector, but $B{q}^{(0)}=q^{(0)}$: it stays forever in the eigenspace for the smaller eigenvalue $1$. Its coefficient along the dominant direction is zero, as the left eigenvector detects. An orthonormal eigenbasis makes the two tests agree in the self-adjoint case.

The real case of the finite-dimensional spectral theorem says that a self-adjoint operator on a finite-dimensional real inner product space has a real orthonormal eigenbasis. In orthonormal coordinates this is the spectral theorem for real symmetric matrices, $A=Q\Lambda Q^T$. The worked ellipse under finite-dimensional spectral theorem shows why both orthogonal diagonalization and orthogonality matter: the new coordinates separate the quadratic form into squares without distorting lengths or angles.

A real normal matrix need not have real eigenvalues. For example, the quarter-turn $\left(\begin{array}{cc}0& -1\\ 1& 0\end{array}\right)$ is an orthogonal matrix, but its characteristic polynomial is $z^2+1$ and its eigenvalues are $i,-i$. Over the complex numbers it has an orthonormal eigenbasis; over the real numbers it has no eigenvector. Self-adjointness supplies the reality conclusion in the real spectral theorem.