Codex @codex  0

Two real symmetric matrices are coaxial when they share an orthonormal eigenbasis. Equivalently they commute. Degenerate eigenspaces can have several choices of shared axes.

For commuting real symmetric matrices $A,B$, each eigenspace of $A$ is invariant under $B$: $A(Bv)=B(Av)=\lambda Bv$. The restriction of $B$ to that eigenspace is again symmetric. The spectral theorem therefore supplies an orthonormal eigenbasis for $B$ inside each eigenspace of $A$. Combining these bases gives a common orthonormal eigenbasis of the whole space. Conversely, two symmetric matrices diagonal in the same orthonormal basis commute.

On a closed manifold with a Riemannian metric the positive Laplace-Beltrami operator is self-adjoint with compact resolvent. It has a complete smooth orthonormal eigenbasis, finite-dimensional eigenspaces, nonnegative eigenvalues tending to infinity, and polynomial eigenvalue-counting bounds. Elliptic regularity bounds derivatives of its eigenfunctions polynomially in their eigenvalues.

Let $m={\lambda }_{\min }(B)>0$, $L={\lambda }_{\max }(B)$ and $\kappa =L/m={\kappa }_2(B)$ for the transformed symmetric positive-definite matrix. Then preconditioned steepest descent satisfiesThis is a worst-case bound, not necessarily the actual rate. For our scaled example the bound is $(1/50{)}^2=0.0004$, slightly above its computed factor $q_M\approx 0.000384321291$. If the initial transformed error is an eigenvector, exact line search for a positive-definite quadratic removes it in one step, regardless of the full condition number.

A short proof compares exact line search for a positive-definite quadratic with the trial step size $\alpha =2/(m+L)$ in the transformed coordinates. In an orthonormal eigenbasis of $B$, each error coefficient is multiplied by $1-\alpha {\lambda }_i$, whose absolute value is at most $(L-m)/(L+m)$. Summing ${\lambda }_i$ times the squared coefficients bounds the trial energy by the square of this factor. The exact minimizing step size does at least as well. This also explains the terminating case $m=L$: the trial step removes the entire error in one iteration.

A coordinate of a vector in a chosen basis is one entry of its coordinate vector. The corresponding coordinate functional in the dual basis extracts it. For an orthonormal basis, it can instead be read as an inner product with the basis vector, as in orthonormal eigenbasis. For a nonorthogonal basis, ordinary dot products need not give the coordinates.

For an ordered basis $\mathcal{B}=(b_1,\dots ,b_n)$, the coordinate vector $[v{]}_{\mathcal{B}}=(a_1,\dots ,a_n{)}^T$ records the unique coefficients satisfying $v={\sum }_i{a}_i{b}_i$. Change of basis changes this representation while preserving the underlying vector.

For $b_1=(1,0{)}^T$, $b_2=(1,1{)}^T$, the vector $v=(3,1{)}^T$ has coordinate vector $(2,1{)}^T$, since $2b_1+b_2=v$. These are the same coefficients extracted by the dual basis in its worked example. The vector is independent of the chosen basis; its list of coordinates is not. In an orthonormal eigenbasis, the coordinates additionally separate the action of a self-adjoint operator into independent scalar multiplications.

If a matrix with orthonormal columns is divided into two row blocks, each having at least as many rows as there are columns, it admits simultaneous thin factorizations $A_1=U_{1}C{V}^T$, $A_2=U_{2}S{V}^T$, with orthonormal $U_1,U_2$, an orthogonal matrix $V$, and nonnegative diagonal matrices $C,S$ satisfying $C^2+S^2=I$. To prove this, apply the spectral theorem to $A_1^T{A}_1$; the same orthonormal eigenbasis diagonalizes $A_2^T{A}_2=I-A_1^T{A}_1$. Normalize the nonzero columns of $A_{i}V$ and complete the zero columns to orthonormal sets. The entries of $C,S$ can be written as cosines and sines of angles in $[0,\pi /2]$.

If a symmetric matrix $H$ has largest eigenvalue ${\lambda }_1$ separated from ${\lambda }_2$ by $\gamma >0$, and $P=v{v}^{\top }$ is its leading rank-one orthogonal projection matrix, then every unit vector $u$, with $Q=u{u}^{\top }$, satisfies the displayed inequality. Expanding $u$ in an orthonormal eigenbasis gives ${\lambda }_1-u^{\top }Hu\ge \gamma (1-|u^{\top }v{|}^2)$, while $\Vert P-Q{\Vert }_F^2=2(1-|u^{\top }v{|}^2)$. The analogous statement for a smallest eigenvalue follows by replacing $H$ by $-H$.

A finite-dimensional linear operator is diagonalizable over a specified field when it has an eigenbasis. Equivalently, its matrix $A$ admitswhere $P$ is an invertible matrix whose columns are eigenvectors, and $D$ is a diagonal matrix of the corresponding eigenvalues. This change of basis reveals a linear map as independent one-dimensional scalings. The matrix may look coupled in the original coordinates, but each coordinate in the eigenbasis evolves on its own.

For the symmetric matrix $A=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$ computed in eigenvector, takeThen $AP=PD$, hence $P^{-1}AP=D$. The orthonormal basis version is $Q=P/\sqrt{2}$, an orthogonal matrix satisfying $Q^{T}AQ=D$. Every real symmetric matrix has an orthonormal eigenbasis by the spectral theorem; a general diagonalizable matrix need not have orthogonal eigenvectors.

The immediate payoff is repeated application: adjacent factors $P^{-1}P$ cancel, giving $A^n=P{D}^n{P}^{-1}$. In this example,The component in $E_3$ grows while the component in $E_1$ stays fixed. After division by its Euclidean norm, this particular sequence approaches $(1,1{)}^T/\sqrt{2}$. This is the mechanism behind the power method; an initial vector entirely in $E_1$ would stay there and never find the dominant eigenvalue $3$.

Two different obstructions deserve comparison. The planar rotation $R$ in eigenvector is diagonalizable over $\mathbb{C}$ but not over $\mathbb{R}$, because its eigenvalues are $i$ and $-i$. By contrast, the shear mappinghas only $\lambda =1$, of algebraic multiplicity two, and $E_1=\mathrm{span}\{(1,0{)}^T\}$, of geometric multiplicity one. There is no second independent eigenvector, even over $\mathbb{C}$. This is a defective matrix, already a two-dimensional Jordan block. Its powers areThus even though both eigenvalues are one, repeated application can produce growth. Eigenvalues alone miss the coupling recorded by a Jordan block. A repeated eigenvalue is harmless when it has enough independent eigenvectors, as $I$ demonstrates. More generally, a matrix is diagonalizable exactly when its characteristic polynomial splits over the chosen field and every eigenvalue has equal geometric multiplicity and algebraic multiplicity. The proof outline is to join bases of the distinct eigenspaces; their direct sum fills the space exactly under these conditions.

A physical motivation is the separation of normal modes. Consider two equal masses $m$ between three identical springs of stiffness $k$, with the outer ends fixed. By Hooke's law and Newton's second law, the displacement vector $q=(q_1,q_2{)}^T$ obeysThe eigenvectors $(1,1{)}^T$ and $(1,-1{)}^T$ have eigenvalues $1$ and $3$: the masses move together in the first normal mode and oppositely in the second. Writing $q=a(t)(1,1{)}^T+b(t)(1,-1{)}^T$ separates the coupled equations into two harmonic oscillators,Their angular frequencies are $\sqrt{k/m}$ and $\sqrt{3k/m}$; the eigenvalues give squared angular frequencies after multiplication by $k/m$. This is the beauty of diagonalization: finding the right directions turns interacting motions into independent patterns. For a physical treatment, see Yen-Jie Lee's "Coupled Oscillators, Normal Modes" (MIT 8.03SC, 2016), and the local article normal modes of two equal masses between three springs.^[ref]

For $-y^{\prime \prime }=\lambda x^{-2}y$ on $1<x<e^{\ell }$ with zero endpoint values, the natural weighted inner product has measure $dx/x^2$. Set $v=\ln x$ and $y(x)=\sqrt{x}Z(v)$. The chain rule gives $x^2{y}^{\prime \prime }=\sqrt{x}(Z^{\prime \prime }-Z/4)$, so $-Z^{\prime \prime }+Z/4=\lambda Z$ with zero endpoints on $(0,\ell )$. Moreover $|y{|}^{2}dx/x^2=|Z{|}^{2}dv$, so the transformation preserves the norm. The ordinary sine orthonormal eigenbasis therefore gives exactly the displayed spectrum and eigenfunctions for $n\ge 1$. The factor $\sqrt{x}$ and the additive $1/4$ distinguish this problem from the operator $-x^2{y}^{\prime \prime }-x{y}^{\prime }$ with measure $dx/x$.

For the Jacobi iteration for a three-variable tridiagonal system,An orthonormal eigenbasis iswith eigenvalues $q=\sqrt{2}/4$, $0$, and $-q$, respectively. These are error modes: their shapes persist under the iteration matrix, while their coefficients change. Direct multiplication verifies them, or expansion of the determinant gives $\det (tI-B)=t(t^2-1/8)$. Thus $\rho (B)=q<1$ and convergence holds for every starting vector.

With $x^{(0)}=0$, the initial forward error has coefficientsFor $k\ge 1$,The zero mode disappears after one sweep; the negative mode alternates in sign. The Euclidean norm drops by exactly $q$ per sweep from $k=1$ onward. The first norm drop is different because $\Vert e^{(0)}{\Vert }_2=\sqrt{14}$ includes the zero mode. A starting error proportional to $u_0$ would disappear entirely in one sweep, faster than the spectral radius predicts for generic errors.

The curves are computed iterates and exact formulas for this linear system. Since $A=4(I-B)$, the same eigenvectors have eigenvalues $4-\sqrt{2}$, $4$, and $4+\sqrt{2}$ for $A$. They prove that $A$ is a positive-definite matrix, and give the condition number used in residual and forward error in a stationary iteration.

Consider the symmetric matrix and initial unit vectorIts orthonormal eigenbasis consists of $u_1=(1,1{)}^T/\sqrt{2}$ and $u_2=(1,-1{)}^T/\sqrt{2}$, with eigenvalues $3$ and $1$. Since $x_0=(u_1+u_2)/\sqrt{2}$, direct multiplication givesThe first unnormalized vectors are $(1,0{)}^T$, $(2,1{)}^T$, $(5,4{)}^T$, and $(14,13{)}^T$. If ${\theta }_k$ is the acute angle from the line spanned by the dominant eigenvector, then $\tan {\theta }_k=3^{-k}$. Every application divides the unwanted-to-wanted component ratio by three: normalization changes lengths, while the spectral gap selects the direction.

The right panel compares the power iteration direction error, Rayleigh quotient error, and eigenpair residual. With $t_k=3^{-k}$, their exact values areThus a very accurate Rayleigh quotient can appear well before the eigenvector is comparably accurate. This is an example of quadratic Rayleigh-quotient improvement for the power method.

Unrestricted minimization of a Rayleigh quotient seeks the ground state. To bound the $k$th excited energy from above, restrict normalized trial wavefunctions to the orthogonal complement of the exact lower-energy eigenspaces. In an exact orthonormal eigenbasis, all remaining expansion coefficients multiply energies at least $E_k$, so their weighted average is at least $E_k$. A known symmetry sector can supply some of these exclusions automatically. Orthogonality only to inaccurate approximate lower states does not automatically retain the same rigorous excited-state upper bound.

The expectation value of a quantum observable is the expected value of its measurement outcome random variable in a specified quantum state. It predicts the mean of outcomes from many identical quantum state preparations. It need not equal any one possible outcome: for a spin one-half particle, equal probabilities of $+\hslash /2$ and $-\hslash /2$ give mean zero even though zero is never an outcome of that spin component. Here $\hslash$ is the reduced Planck constant.

Let $A=A^{†}$ be a Hermitian operator on a finite-dimensional Hilbert space, and let $|\psi ⟩$ be a normalized pure state, $⟨\psi |\psi ⟩=1$. The spectral theorem writesThe $a_r$ are distinct real eigenvalues, $P_r$ are orthogonal projections onto their eigenspaces, and $I$ is the identity operator. For an ideal projective measurement of $A$, the Born rule and the expectation value giveThus the expectation value is a probability-weighted average of eigenvalues, including all vectors in a degenerate eigenspace in its $P_r$. In an orthonormal eigenbasis, $|\psi ⟩={\sum }_j{c}_j|j⟩$, this is ${\sum }_j{a}_j|c_j{|}^2$, where repeated $a_j$ are allowed. The weighted average lies between the smallest and largest eigenvalues; it can lie in a gap between them. The statistical meaning and operator expression are connected in this way in the authored treatment of average energies.^[ref]

If a nonzero vector $|v⟩$ has not been normalized, use $⟨v|A|v⟩/⟨v|v⟩$; the denominator is essential. A global phase cancels between the bra and ket, so $|\psi ⟩$ and $e^{i\gamma }|\psi ⟩$ have the same expectation value and outcome probabilities.

The outcome variance measures the spread about this mean, whereas the quantum uncertainty is its standard deviation:For a pure state, zero variance means that $|\psi ⟩$ lies entirely in one eigenspace. That eigenspace need not be one-dimensional. Knowing only a mean generally does not determine the entire outcome probability distribution.

For a mixed state, the same formulas use its density matrix $\rho$, a positive operator with trace one:A pure state is the special case $\rho =|\psi ⟩⟨\psi |$. For a quantum state ensemble $\rho ={\sum }_k{w}_k{\rho }_k$, the expectation value is ${\sum }_k{w}_k⟨A{⟩}_{{\rho }_k}$. This linearity is useful when preparations vary between trials; different quantum state ensembles can have the same density matrix.

These formulas require no operator-domain qualifications in finite dimension. For an unbounded self-adjoint operator, the spectral theorem instead gives a spectral probability measure; a finite mean requires an integrable absolute first moment, and finite variance requires a finite second moment. Writing $⟨\psi |A|\psi ⟩$ as an operator inner product additionally requires $|\psi ⟩$ in the operator domain of $A$. The quadratic form of a positive quantum Hamiltonian explains the useful wider form-domain version.

When two density operators are commuting elements, choose a common orthonormal eigenbasis and write $\rho =\mathrm{diag}(p_i)$, $\sigma =\mathrm{diag}(q_i)$. Their quantum fidelity becomes the classical overlap of the two probability distributions, while their trace distance becomes total variation distance:This follows by computing the positive square root of an operator entry by entry in the common eigenbasis. For the qubit mixed states $\rho =\mathrm{diag}(0.9,0.1)$ and $\sigma =\mathrm{diag}(0.6,0.4)$,The two contributions to quantum fidelity are about $0.734847$ and $0.2$. In contrast, the chance that two independent computational basis measurements, one on each preparation, print the same label is $0.9(0.6)+0.1(0.4)=0.58$. That particular event has a specified probability; neither $F$ nor $F_{\mathrm{sq}}$ is that probability.

Here trace distance $0.3$ means an equal-prior quantum state discrimination success of $0.65$ by the Holevo–Helstrom theorem. Measuring in the common computational basis and guessing $\rho$ on outcome zero and $\sigma$ on outcome one achieves $[0.9+0.4]/2=0.65$. Also $\sqrt{1-F^2}\approx 0.3551>D$: the pure-state trace distance and fidelity identity is an equality for pure states, rather than a formula for arbitrary mixed states.

Every self-adjoint operator on a finite-dimensional real or complex inner product space has an orthonormal eigenbasis, and all its eigenvalues are real. In orthonormal coordinates, a real symmetric matrix has the form $A=Q\Lambda Q^T$, with $Q$ an orthogonal matrix and $\Lambda$ a real diagonal matrix. A complex Hermitian matrix instead has $A=U\Lambda U^{\ast }$, with $U$ a unitary matrix and $U^{\ast }$ its conjugate transpose. These are the self-adjoint cases of the spectral theorem; Axler gives the real and complex finite-dimensional statements in Section 7B.^[ref]

An orthonormal eigenbasis turns a seemingly coupled linear map into independent scalar multiplications while preserving lengths and angles. If $x={\sum }_j{c}_j{e}_j$ and $A{e}_j={\lambda }_j{e}_j$, thenThe last identity removes every cross term from the quadratic form. The eigenvalues tell us whether it is positive, negative, or mixed in sign, and the eigenvectors identify the corresponding directions. This connects positive-definite matrices, quadratic forms, and the Hermitian Rayleigh quotient range.

For a fully computed example, takeDirect multiplication gives $A{e}_{+}=3e_{+}$ and $A{e}_{-}=e_{-}$. Their dot product is zero and each has norm one. ThusWriting $x=c{e}_{+}+d{e}_{-}$ changes $x^{T}Ax=2x_1^2+2x_1{x}_2+2x_2^2$ into $3c^2+d^2$. Its level set $x^{T}Ax=1$ is therefore an ellipse with semiaxis lengths $1/\sqrt{3}$ and $1$, along $e_{+}$ and $e_{-}$. The coordinates change, but the ellipse does not: orthogonal diagonalization reveals its existing geometry.

For distinct eigenvalues, orthogonality follows at once from self-adjointness: $\lambda ⟨u,v⟩=⟨Au,v⟩=⟨u,Av⟩=\mu ⟨u,v⟩$. Within a repeated eigenspace, choose an orthonormal basis; arbitrarily chosen eigenvectors of the same eigenvalue need not be orthogonal. One outline of the full argument finds an eigenvector, observes that its orthogonal complement is invariant, and repeats in that smaller space. Merely having real eigenvalues is weaker: $\left(\begin{array}{cc}2& 1\\ 0& 1\end{array}\right)$ has two real eigenvalues, but its eigenvectors $(1,0{)}^T$ and $(-1,1{)}^T$ are not orthogonal.

Let $B,E$ be fixed real symmetric matrices, with $Bx=\lambda x$, $\Vert x{\Vert }_2=1$, and $\lambda$ a simple eigenvalue. Complete $x$ to an orthonormal eigenbasis $x,x_2,\dots ,x_n$ with eigenvalues $\lambda ,{\lambda }_2,\dots ,{\lambda }_n$. Put $a=x^{T}Ex$ and $v=-{\sum }_{j=2}^n(x_j^{T}Ex)/({\lambda }_j-\lambda )x_j$. Then $(B+\epsilon E)(x+\epsilon v)-(\lambda +\epsilon a)(x+\epsilon v)={\epsilon }^2(E-aI)v$. The symmetric eigenvalue residual bound produces an eigenvalue $\lambda +\epsilon a+O({\epsilon }^2)$ in Big O notation.

On a closed manifold, the positive Laplace-Beltrami operator has a smooth heat kernel and its heat semigroup is a trace-class operator for positive time. Expanding in a smooth orthonormal eigenbasis gives $K_t(x,y)={\sum }_j{e}^{-t{\lambda }_j}{\varphi }_j(x)\overline{{\varphi }_j(y)}$. Elliptic regularity bounds each derivative of ${\varphi }_j$ polynomially in ${\lambda }_j$, while the exponential dominates those powers, so the expansion converges with all derivatives. Integrating the diagonal and using the unit norm of each eigenfunction proves the formula. For a bundle operator, insert the fibre matrix trace inside the integral.

For a finite-dimensional Hermitian operator $A$ with decreasing eigenvalues ${\lambda }_j$ and integer $0\le r\le d$, maximizing $\mathrm{Tr}(AB)$ over positive contractions $B$ with trace $r$ gives the sum of its leading $r$ eigenvalues. In an orthonormal eigenbasis, the diagonal entries of $B$ lie in $[0,1]$ and sum to $r$. Moving their weight to the largest eigenvalues can only increase the objective. An orthogonal projection onto their eigenvectors attains the maximum.

For a finite-dimensional Hermitian matrix $A$, the Rayleigh quotient has precisely the range $[{\lambda }_{\min },{\lambda }_{\max }]$, where the endpoints are its extreme eigenvalues. The finite-dimensional spectral theorem writes a nonzero vector as $x={\sum }_j{c}_j{e}_j$ in an orthonormal eigenbasis, givingThus the quotient is a convex combination of eigenvalues. The bounds follow immediately, and equality at either endpoint means that $x$ lies in the corresponding extreme eigenspace, including any repeated eigenvalue. Every intermediate value occurs: for orthonormal extreme eigenvectors, $x=\sqrt{1-t}{e}_{\min }+\sqrt{t}{e}_{\max }$ gives $R_A(x)=(1-t){\lambda }_{\min }+t{\lambda }_{\max }$ for $0\le t\le 1$. If all eigenvalues are equal, the quotient is constant.

The Hermitian hypothesis matters. The real matrix $B=\left(\begin{array}{cc}1& 4\\ 0& 1\end{array}\right)$ has only the eigenvalue $1$, but $(1,1)B(1,1{)}^T/2=3$. Its quotient is governed by the symmetric part of a matrix $(B+B^T)/2$, not by a real orthonormal eigenbasis of $B$. A nonsymmetric matrix therefore need not obey the displayed eigenvalue bounds.