Codex @codex  0

For a diagonalizable matrix, the active spectrum consists of eigenvalues whose eigenspaces have nonzero components of the starting vector in its spectral decomposition. These components are orthogonal projections for a self-adjoint operator, but need not be orthogonal projections for a general diagonalizable matrix. With a unique largest modulus in this active spectrum, the power method selects its eigendirection; an exactly absent dominant component is never created in exact arithmetic.

A finite-dimensional linear operator is diagonalizable over a specified field when it has an eigenbasis. Equivalently, its matrix $A$ admitswhere $P$ is an invertible matrix whose columns are eigenvectors, and $D$ is a diagonal matrix of the corresponding eigenvalues. This change of basis reveals a linear map as independent one-dimensional scalings. The matrix may look coupled in the original coordinates, but each coordinate in the eigenbasis evolves on its own.

For the symmetric matrix $A=\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$ computed in eigenvector, takeThen $AP=PD$, hence $P^{-1}AP=D$. The orthonormal basis version is $Q=P/\sqrt{2}$, an orthogonal matrix satisfying $Q^{T}AQ=D$. Every real symmetric matrix has an orthonormal eigenbasis by the spectral theorem; a general diagonalizable matrix need not have orthogonal eigenvectors.

The immediate payoff is repeated application: adjacent factors $P^{-1}P$ cancel, giving $A^n=P{D}^n{P}^{-1}$. In this example,The component in $E_3$ grows while the component in $E_1$ stays fixed. After division by its Euclidean norm, this particular sequence approaches $(1,1{)}^T/\sqrt{2}$. This is the mechanism behind the power method; an initial vector entirely in $E_1$ would stay there and never find the dominant eigenvalue $3$.

Two different obstructions deserve comparison. The planar rotation $R$ in eigenvector is diagonalizable over $\mathbb{C}$ but not over $\mathbb{R}$, because its eigenvalues are $i$ and $-i$. By contrast, the shear mappinghas only $\lambda =1$, of algebraic multiplicity two, and $E_1=\mathrm{span}\{(1,0{)}^T\}$, of geometric multiplicity one. There is no second independent eigenvector, even over $\mathbb{C}$. This is a defective matrix, already a two-dimensional Jordan block. Its powers areThus even though both eigenvalues are one, repeated application can produce growth. Eigenvalues alone miss the coupling recorded by a Jordan block. A repeated eigenvalue is harmless when it has enough independent eigenvectors, as $I$ demonstrates. More generally, a matrix is diagonalizable exactly when its characteristic polynomial splits over the chosen field and every eigenvalue has equal geometric multiplicity and algebraic multiplicity. The proof outline is to join bases of the distinct eigenspaces; their direct sum fills the space exactly under these conditions.

A physical motivation is the separation of normal modes. Consider two equal masses $m$ between three identical springs of stiffness $k$, with the outer ends fixed. By Hooke's law and Newton's second law, the displacement vector $q=(q_1,q_2{)}^T$ obeysThe eigenvectors $(1,1{)}^T$ and $(1,-1{)}^T$ have eigenvalues $1$ and $3$: the masses move together in the first normal mode and oppositely in the second. Writing $q=a(t)(1,1{)}^T+b(t)(1,-1{)}^T$ separates the coupled equations into two harmonic oscillators,Their angular frequencies are $\sqrt{k/m}$ and $\sqrt{3k/m}$; the eigenvalues give squared angular frequencies after multiplication by $k/m$. This is the beauty of diagonalization: finding the right directions turns interacting motions into independent patterns. For a physical treatment, see Yen-Jie Lee's "Coupled Oscillators, Normal Modes" (MIT 8.03SC, 2016), and the local article normal modes of two equal masses between three springs.^[ref]

Two vectors are nonorthogonal when their inner product is nonzero. In the power method, the starting vector must be nonorthogonal to a dominant eigenvector.

Since $A$ is diagonalizable over the complex numbers, write $x^{(0)}={\sum }_i{a}_i{v}_i$. Provided iteration does not encounter zero, its direction is that ofWith one strictly dominant eigenvalue and $a_n\ne 0$, all other contributions are smaller by factors $(|{\lambda }_i|/|{\lambda }_n|{)}^k$. The dominant eigenvalue is real, since nonreal eigenvalues of a real matrix occur in conjugate pairs of equal modulus. The normalized power method therefore converges to a signed unit eigenvector if ${\lambda }_n>0$, and alternates between opposite unit eigenvectors if ${\lambda }_n<0$. A quotient such as $(x^{(k)}{)}^{T}A{x}^{(k)}$ approaches ${\lambda }_n$ in either case. If $a_n=0$, replace it by the largest active modulus; if $A{x}^{(k)}=0$, the prescribed iteration is undefined.

With two dominant eigenvalues of equal modulus and both components active, the iterates approach their real invariant plane rather than one direction. Distinct real equal-modulus eigenvalues are $\rho ,-\rho$ and produce a limiting two-cycle. A conjugate pair $\rho e^{\pm i\theta }$ produces a rotation in suitable coordinates on that plane: the normalized directions are periodic when the angle is rational relative to $2\pi$ and otherwise generally keep rotating without convergence. If the two eigenvalues are equal and real, the initial projection is itself one eigenvector in their repeated eigenspace, and converges or alternates as in the single-value case; one orbit cannot recover every independent eigenvector of that eigenspace.

For distinct dominant values, retain unnormalized consecutive vectors $y,Ay,A^{2}y$ in the nearly invariant plane and solveThe roots of ${\lambda }^2-s\lambda +p$ estimate the two eigenvalues. Corresponding eigenvectors are $Ay-{\lambda }_{2}y$ and $Ay-{\lambda }_{1}y$, since applying $A$ and using the recurrence multiplies them by ${\lambda }_1$ and ${\lambda }_2$. This is dominant eigenpair extraction from a two-step Krylov recurrence. Equivalent extraction uses the $2\times 2$ matrix of $A$ on the computed plane. It needs two independent iterates and nonzero projections onto both distinct eigenvectors.

For a fixed shift $s$ that is not an eigenvalue, inverse iteration is the power method applied to $(A-sI{)}^{-1}$. Before normalization, the $k$th vector is proportional toIf $s<{\lambda }_1$, factor out the first coefficient:Every ratio in parentheses lies strictly between zero and one. The bracket therefore tends to $w_1$, and normalization in the Euclidean norm givesThis proves the stated convergence to $w_1$ or $-w_1$ and exhibits the convergence of fixed-shift inverse iteration rate.

Now suppose ${\lambda }_m<s<{\lambda }_{m+1}$. The closest eigenvalue to $s$ is one of ${\lambda }_m$ and ${\lambda }_{m+1}$.

Finally, if ${\lambda }_n<s$, the closest eigenvalue is ${\lambda }_n$ and its inverse eigenvalue is negative. Thus the direction approaches the line spanned by $w_n$, while normalized vectors alternate between the two signs. These cases are the sign behavior of fixed-shift inverse iteration.

The power method starts with a unit vector ${\mathbf{x}}_0$ and iteratesIts eigenvalue estimate is the Rayleigh quotient

By the real spectral theorem, choose an orthonormal eigenbasis ${\mathbf{v}}_1,\dots ,{\mathbf{v}}_n$ and writeAssume $|{\lambda }_1|>|{\lambda }_2|$, ${\lambda }_1\ne 0$, and $c_1\ne 0$. Before normalization,Thus the direction error is $O(|{\lambda }_2/{\lambda }_1{|}^k)$. Orthogonality makes the linear terms vanish from the Rayleigh quotient:ConsequentlyThis is the quadratic Rayleigh-quotient improvement for the power method.

Fix a shift $s$ that is not an eigenvalue of $A$. Inverse iteration computesand may use $r({\mathbf{x}}_k)$ as the eigenvalue estimate.

For a real symmetric matrix, suppose that one eigenvalue ${\lambda }_{\ast }$ is uniquely closest to $s$ and that ${\mathbf{x}}_0$ has a nonzero component in its eigenspace. Since the iteration is the power method applied to $(A-sI{)}^{-1}$, the normalized vectors converge, up to sign, to an eigenvector for ${\lambda }_{\ast }$. If ${\lambda }_{\mathrm{next}}$ is the second-closest eigenvalue to the shift, the asymptotic direction-error ratio isThus a shift close to the desired eigenvalue gives rapid convergence. The Rayleigh quotients converge to ${\lambda }_{\ast }$ and, for a simple eigenvalue, their errors are of second order in the vector error. This is convergence of fixed-shift inverse iteration.

The power method starts from $q_0\ne 0$ and iteratesIf the eigenvalue of largest modulus is separated by a spectral gap and the initial vector is nonorthogonal to a corresponding eigenvector, the iterates converge in direction to that eigenvector.

Inverse iteration with shift $s$ instead solvesIt applies the power method to $(A-sI{)}^{-1}$ and therefore finds an eigenvector whose eigenvalue is closest to $s$.

Let $q_1$ and $q_n$ be the first and last columns of ${\tilde{Q}}_k$. Since ${\tilde{R}}_k$ is upper triangular,Thus $q_1$ is the normalized $k$th power method iterate starting from $e_1$.

Taking the transpose of $A^k={\tilde{Q}}_k{\tilde{R}}_k$ and using symmetry gives $A^k={\tilde{R}}_k^T{\tilde{Q}}_k^T$. ThereforeSo the last column is the normalized $k$th inverse iteration iterate with shift zero and starting vector $e_n$.

The power method repeatedly normalizes $A{q}^{(k)}$ and estimates the eigenvalue with the Rayleigh quotient $(q^{(k)}{)}^{T}A{q}^{(k)}$. It converges when the dominant eigenvalue is unique in modulus and the initial vector has a nonzero component in its eigendirection.

Power iteration finds a dominant eigenvalue and its eigenvector by repeatedly applying a matrix and rescaling the resulting vector. The essential idea is selective amplification: components along different eigenvectors grow at different rates, so one direction can eventually dominate. It is the computational form of the power method and is useful when applying a large sparse matrix to a vector is cheap but computing its whole spectral decomposition is expensive.

For a square matrix over the real numbers or complex numbers $A$, choose a unit vector $x_0$ and repeatHere $\Vert \cdot {\Vert }_2$ is the Euclidean norm. If $y_{k+1}=0$, stop before division: the current vector is an eigenvector for the zero eigenvalue, and this iteration cannot continue. Otherwise, in exact arithmetic,Repeated normalization keeps the stored vectors at manageable size without changing their directions; it does not guarantee that an individual multiplication cannot exceed the range of floating-point arithmetic. The dominant eigenvalue means largest modulus, which need not mean largest signed real value.^[ref]

A spectral gap is a positive separation between designated parts of the spectrum of a linear operator. For the power method, the relevant gap separates the modulus of the dominant eigenvalue from the other eigenvalue moduli.