Codex @codex  0

For positive-definite within-group scatter $W$ and between-group scatter $B$, directions maximizing $a^{T}Ba/(a^{T}Wa)$ solve the displayed generalized eigenvalue problem. Put $u=W^{1/2}a$; the quotient becomes the Rayleigh quotient of $W^{-1/2}B{W}^{-1/2}$. Its ordered orthonormal eigenvectors give directions orthogonal in the $W$ inner product. There are at most $g-1$ nonzero directions, where $g$ is the number of groups. These directions summarize separation for linear discriminant analysis.

For $-(p{y}^{\prime }{)}^{\prime }=\lambda wy$ on $[0,1]$ with $p=(1+x)e^{2x}$, $w=e^{2x}$, $y(0)=0$ and $y^{\prime }(1)=0$, the Rayleigh quotient is $\int p{y}^{\prime 2}/\int w{y}^2$. The trial $y_1=x{e}^{-x}$ gives numerator $5/12$ and denominator $1/3$, hence $0<{\lambda }_{\min }\le 5/4$. Every displayed $y_n$ obeys the same mixed boundary conditions; so does $\sin [(m+1/2)\pi x]$. Adding any of these to the trial space cannot worsen the optimized upper bound. Matrix elements $A_{ij}=\int p{y}_i^{\prime }{y}_j^{\prime }$ and $M_{ij}=\int w{y}_i{y}_j$ give a finite-dimensional generalized eigenvalue problem $Av=\lambda Mv$. The case $n=1$ repeats the original trial and adds no variational freedom.

For symmetric $A$ and positive-definite $B$, the generalized Rayleigh quotient is $x^{T}Ax/(x^{T}Bx)$. Its stationary values are the generalized eigenvalues of $Av=\lambda Bv$, and its maximum is the largest such eigenvalue.

For one localized orbital per site in a chain, take on-site matrix elements $H_{jj}=E_0$, nearest-neighbour matrix elements $H_{j,j\pm 1}=h$, overlaps $S_{jj}=1$ and $S_{j,j\pm 1}=s$, with other terms neglected and $h,s$ real. A Bloch state reduces the generalized eigenvalue problem $Hc=ESc$ toThe overlap metric must be positive for the retained model; $|s|<1/2$ is sufficient for all $k$. Dropping it yields the cosine band of the orthogonal tight-binding approximation.

For equal point masses and upper/lower massless rods of length $2l,l$, angular coordinates from the downward vertical give mass matrix $m{l}^2\left(\begin{array}{cc}8& 2\\ 2& 1\end{array}\right)$ and stiffness matrix $mgl\mathrm{diag}(4,1)$. The generalized eigenvalue problem reduces to $s^2-3s+1=0$, where $s={\omega }^{2}l/g$. The slow mode has angular-amplitude ratio ${\theta }_2/{\theta }_1=\sqrt{5}-1$, and the fast mode has ratio $-1-\sqrt{5}$.

For a uniform rigid beam of mass $m$ and two identical vertical end springs of stiffness $k$, small vertical end displacements have kinetic energy $m({\dot{x}}_1^2+{\dot{x}}_1{\dot{x}}_2+{\dot{x}}_2^2)/6$ and displacement-dependent potential energy $k(x_1^2+x_2^2)/2$. The mass and stiffness matrices are $\mathsf{M}=(m/6)\left(\begin{array}{cc}2& 1\\ 1& 2\end{array}\right)$ and $\mathsf{K}=k1$. Their generalized eigenvalue problem gives symmetric translation $(1,1)$ and antisymmetric rocking $(1,-1)$ at the displayed angular frequencies. The centre coordinate contains only the symmetric mode. Neglect spring inertia, damping and finite-angle geometric corrections.

A cart of mass $m$ moves horizontally on a spring of stiffness $k$, carrying a simple pendulum of bob mass $M$ and length $L$. Let $x$ be cart displacement and $\varphi$ the small angle from downward vertical. The quadratic Lagrangian isThe kinetic cross term comes from the bob's horizontal velocity $\dot{x}+L\dot{\varphi }$ to first order. The generalized eigenvalue problem has mass matrix $\left(\begin{array}{cc}m+M& ML\\ ML& M{L}^2\end{array}\right)$ and restoring matrix $\mathrm{diag}(k,MgL)$. Their characteristic determinant gives the displayed polynomial in ${\omega }^2$. For $m=M=L=g=1,k=2$, the squared angular frequencies are $2\mp \sqrt{2}$, with mode shapes $(x,\varphi )=(1,\pm \sqrt{2})$. The bob and cart move in phase in the lower normal mode and in opposite phases in the higher normal mode.

Consider three beads of masses $m,m,M$ on a horizontal hoop, connected in cyclic order by identical springs constrained to follow the arcs. Their common natural length affects only a constant in the potential energy, since the sum of the three arc lengths is fixed. About the equally spaced configuration, angular displacements $q_i$ haveThe generalized eigenvalue problem has the rigid rotation $(1,1,1)$ as a zero mode. Exchanging the equal masses separates an antisymmetric normal mode $(1,-1,0)$, with squared angular frequency $3k/m$, from a symmetric normal mode $(1,1,-2m/M)$, with squared angular frequency $k/m+2k/M$. The last mode shape has zero total angular momentum. The hoop radius cancels from both the kinetic energy and the restoring quadratic form. These formulas use arc-constrained springs; straight chord springs have a different potential energy.

The quantum variational principle turns the ground-state problem into minimization of a Rayleigh quotient:The Hamiltonian must be a self-adjoint operator bounded below, and the nonzero trial state must have finite form energy. Expanding a normalized trial state in exact energy eigenstates gives $E[\psi ]={\sum }_n|c_n{|}^2{E}_n$, a weighted average no smaller than $E_0$. Equality requires support only in the ground-state eigenspace. The spectral version also accommodates continuous spectrum; a lowest normalizable eigenstate is not required merely to bound the spectral infimum.

The variational method chooses a trial family with the correct boundary conditions, fermionic exchange symmetry and other required quantum numbers, evaluates its energy and minimizes over its parameters. It gives an upper bound on ground-state energy. Enlarging a nested trial family can only improve or preserve its optimal bound. A low energy does not by itself establish a highly accurate wavefunction, especially near degeneracy; energy errors are second order in small admixtures orthogonal to a nondegenerate exact ground state.

For example, a normalized Gaussian trial for a one-dimensional quantum harmonic oscillator is ${\psi }_{\beta }=(\beta /\pi {)}^{1/4}{e}^{-\beta x^2/2}$, with $\beta >0$. Its energy isThe two terms balance kinetic localization cost against the potential cost of spreading out. This Gaussian variational energy for the harmonic oscillator happens to attain the exact answer because the family contains the exact ground state.

A trial expansion $\psi ={\sum }_j{c}_j{\chi }_j$ in a finite, possibly nonorthogonal basis gives the generalized eigenvalue problem $H\mathbf{c}=ES\mathbf{c}$, with $H_{ij}=⟨{\chi }_i|H|{\chi }_j⟩$ and $S_{ij}=⟨{\chi }_i|{\chi }_j⟩$. Its lowest Ritz value minimizes the quotient in that subspace. For excited states, the excited-state variational constraint imposes exact orthogonality to the lower eigenstates to obtain a corresponding upper bound; the Courant–Fischer min-max principle yields upper bounds for ordered Ritz eigenvalues from a common subspace. orthogonality only to an independently approximate ground state does not automatically provide an excited-state bound: residual true-ground-state contamination can lower the trial energy below the desired excited energy. Symmetry can impose exact orthogonality when the relevant states have different symmetry sectors.

Set the support displacement to zero and seek a normal mode $\mathbf{x}=(A,B{)}^{\mathsf{T}}\cos (\omega t+\phi )$. With $s=m{\omega }^2/k$, the generalized eigenvalue problem requiresThusThe second row gives $A=(1-s)B$. The signed displacement ratios are thereforeThe slow mode is in phase and the fast mode out of phase. Keeping the unequal mass matrix is essential: replacing it by the identity changes the eigenfrequencies.

For the normal modes of a double pendulum with a two-to-one length ratio, seek $\mathbf{\theta }=\mathbf{a}\cos (\Omega t+\phi )$. The generalized eigenvalue problem is $(\mathbf{V}-{\Omega }^2\mathbf{T})\mathbf{a}=0$. Put $s={\Omega }^{2}l/g$. A nonzero mode requiresThus the two angular frequencies areBoth squared frequencies are positive. The slower mode has ${\theta }_2/{\theta }_1=\sqrt{5}-1$ and is in phase; the faster mode has ${\theta }_2/{\theta }_1=-1-\sqrt{5}$ and is out of phase. These mode ratios also check the off-diagonal terms in the mass matrix.

Write the displacement as $\mathbf{x}=\mathbf{a}\cos (\omega t+\varphi )$. The generalized eigenvalue problem isThe symmetry of the two end atoms suggests an end-antisymmetric shape $(1,0,-1)$ and two end-symmetric shapes. Direct substitution gives the normal modes of three masses connected by two equal springs:These three independent eigenvectors exhaust the three-dimensional system. The ordinary frequencies are $\omega /(2\pi )$; the values $\omega$ above are angular frequencies.

To find the normal modes of two arc-spring-coupled beads on a hoop, take $x_i=a_i{e}^{i\Omega t}$ for a normal mode. The resulting generalized eigenvalue problem isIts determinant isThus the angular frequencies areThe ordinary nonzero frequency is $\omega /(2\pi )$. The zero mode has mode shape $(1,1)$ and represents a common rotation with no change in spring length. The oscillatory mode shape is $(1,-1/q)$.

The within-group and between-group scatter decomposition separates noise from group separation: $W$ measures variation about each group sample mean, and $B$ measures separation of those means about the overall sample mean. In linear discriminant analysis, seek a projection $a^{T}x$ maximizing between-group relative to within-group variation,Normalize by $a^{T}Wa=1$; a Lagrange multiplier calculation gives the generalized eigenvalue problem $Ba=\lambda Wa$. The leading canonical discriminant directions have the largest eigenvalues, and subsequent directions are orthogonal in the $W$ inner product. There are at most $\min (p,g-1)$ nonzero separating directions. Projecting observations onto these directions gives a low-dimensional display of group separation.

For classification, estimate the common covariance matrix by $S_p=W/(n-g)$ and use the linear discriminant analysis scoreswhere ${\pi }_{\nu }$ are class prior probabilities. Assign to the largest score. The same within-group scatter sets the noise scale, while between-group scatter selects the directions in which the group means separate most strongly.

Multivariate analysis of variance extends analysis of variance to vector responses. For example, a treatment may affect several correlated physiological measurements jointly. Separate univariate tests do not use the correlation geometry and require adjustment if they are interpreted as one multiple-testing procedure. A joint test asks whether group mean vectors differ in any direction.

In a one-way model, take independent $Y_{ri}\sim N_p({\mu }_r,\Sigma )$, $r=1,\dots ,g$, $i=1,\dots ,n_r$, with a common positive-definite covariance. Put $N={\sum }_r{n}_r$, and let ${\bar{Y}}_r$ and $\bar{Y}$ be group and overall sample means. The within-group and between-group scatter matrices areExpanding around each group mean makes the cross terms vanish, giving total scatter $E+H$. To test $H_0:{\mu }_1=\cdots ={\mu }_g$, use the Wilks lambda statisticwhere ${\theta }_j$ solve the generalized eigenvalue problem $Hv=\theta Ev$ and $E$ is nonsingular. Under the alternative, maximize the normal likelihood at the group means and covariance $E/N$; under the null, use the overall mean and covariance $(E+H)/N$. Their likelihood ratio is ${\Lambda }^{N/2}$, so small $\Lambda$ rejects equal means. Under the null, orthogonal projections of the Gaussian data give independent $E\sim W_p(\Sigma ,N-g)$ and $H\sim W_p(\Sigma ,g-1)$, supplying its null calibration. Here $E$ is nonsingular almost surely when $N-g\ge p$. Merely having more total observations than responses is not enough if group fitting consumes too many statistical degrees of freedom.

The test is invariant under a nonsingular linear change of response coordinates: both determinants acquire the same factor $|A{|}^2$. The directions associated with large ${\theta }_j$ identify mean separation relative to residual variability. With one response it becomes the ordinary ANOVA F-test. With two groups, writing $S_p=E/(N-2)$ and $d={\bar{Y}}_1-{\bar{Y}}_2$, the version using Hotelling's T-squared statistic iswith $N>p+1$. This formula demonstrates explicitly how the covariance weights a group difference.

The sketch shows hypothetical groups with strongly positively correlated responses and mean separation along their low-variance contrast. Marginal comparisons can look weak even when the joint comparison is informative. The ellipses represent individual-response covariance, not confidence regions for the means. Independence of observations, approximate multivariate normality and common covariance are substantive assumptions; outliers and unequal covariance can invalidate the usual calibration. Follow-up contrasts can identify the affected responses or groups, with appropriate multiplicity control. MANOVA uses shared covariance to test a multivariate mean difference, rather than testing unrelated coordinates in isolation.

Varying the Lagrangian gives, with the cyclic angle conventions,All successive angular gaps are $2\pi /N$ in equilibrium, so their two contributions cancel. Because the potential depends only on gaps, a uniform change of all angles costs no energy. Its Hessian therefore annihilates $(1,\dots ,1)$, giving a zero-frequency normal mode. The corresponding motion is rigid rotation, ${\theta }_i(t)={\theta }_i^{(0)}+A+Bt$, including a constant common angular shift.

For two particles write ${\theta }_2-{\theta }_1=\pi +{\delta }_2-{\delta }_1$. Both cyclic gaps contribute, so the quadratic potential is $k({\delta }_2-{\delta }_1{)}^2$. With the specified masses, the mass matrix is $km\mathrm{diag}(1,2)$ and the stiffness matrix is $2k\left(\begin{array}{cc}1& -1\\ -1& 1\end{array}\right)$. The generalized eigenvalue problem givesThe finite-frequency normal mode keeps the mass-weighted mean angle fixed and changes the angular separation sinusoidally. Together with the two rigid-rotation constants, it spans all small motions about equilibrium.

The eigenvalue stability analysis of a finite difference method reduces a linear evolution discretization to its modal behavior, but it must retain the norm and the dependence on the mesh. For a homogeneous system obtained by the method of lines, $U^{\prime }=A_{h}U$, the solution is $e^{t{A}_h}U(0)$. A one-step time discretization gives $U^{n+1}=G_{h,k}{U}^n$; for a Runge-Kutta method, where its stage matrices are invertible, $G_{h,k}=R(k{A}_h)$. With a multistep method one instead uses an amplification matrix on the augmented vector of several time levels.

The relevant finite-time stability estimate iswith $C_T$ independent of the refining spatial mesh and allowed time steps. Similarly, semidiscrete stability requires $\Vert e^{t{A}_h}{\Vert }_h\le C_T$ for $0\le t\le T$. The norm should represent the continuum problem, such as the discrete L2 norm $\Vert U{\Vert }_{2,h}^2=h{\sum }_j|U_j{|}^2$ in one dimension. A fixed finite matrix having decaying solutions as $t\to \infty$ is not by itself a mesh-uniform result for the partial differential equation.

If $G=X\Lambda X^{-1}$, thenThus eigenvalues of modulus at most one give a uniform bound when the diagonalizing bases have uniformly bounded condition numbers. For a normal matrix in the chosen inner product, the spectral theorem for normal operators supplies an orthonormal basis, so the condition number is one and the norm of $G^n$ is exactly ${\max }_j|{\lambda }_j{|}^n$. For a self-adjoint or skew-adjoint spatial matrix, applying a scalar stability function preserves this favorable modal structure. This is the setting where eigenvalue calculations provide especially clean, reliable step restrictions.

A precise fixed-matrix statement is also useful: a matrix is power bounded for all nonnegative integers $n$ if and only if all its eigenvalues lie in the closed unit disk and every Jordan block at a unit-modulus eigenvalue has size one. Indeed, powers of an interior Jordan block contain a polynomial in $n$ times a decaying geometric factor and are bounded; a larger block on the unit circle produces unbounded polynomial growth. For a family of matrices, the resulting bounds must still be uniform. Moreover, finite-time stability versus power boundedness distinguishes this all-time condition from the physical-time estimate above. For example $G_k=I+kN$, $N^2=0$, has $G_k^n=I+nkN$, bounded for $nk\le T$ despite a nontrivial Jordan block at one. Even the scalar update $1+k$ is stable on fixed intervals, since $(1+k{)}^n\le e^T$, although it represents a growing equation rather than a contraction.

For a constant-coefficient periodic difference operator, Fourier modes diagonalize the spatial matrix. The discrete Fourier transform is a unitary operator, so Parseval's identity identifies its modal maximum with the operator norm. The eigenvalue stability analysis of a finite difference method then becomes von Neumann stability analysis. For the Dirichlet discrete Laplacian, a discrete sine transform plays the same role. These facts explain its advantages: a large matrix calculation becomes a scalar symbol or a known spectral interval, and the extremal eigenvalue directly determines a safe time step.

As a first example, centered space discretization of the heat equation $u_t=u_{xx}$ has eigenvaluesForward Euler method gives $R(k\lambda )=1-4r{\sin }^2(\theta /2)$ with $r=k/h^2$. Requiring all factors in $[-1,1]$ yields $0\le r\le 1/2$ for all periodic meshes. In two dimensions the corresponding uniform bound is $r\le 1/4$. Because these are normal matrices, these scalar bounds prove contraction rather than merely suggest it.

Backward Euler method has $R(z)=1/(1-z)$, so the centered discretization of the heat equation is contractive for every $k>0$. The Crank-Nicolson method has $R(z)=(1+z/2)/(1-z/2)$ and is likewise contractive for these negative real modes. However, its factor tends to $-1$ for very stiff modes, leaving high-frequency oscillations weakly damped; Backward Euler method tends to zero. The eigenvalues therefore expose the presence of stiff differential equations and damping as well as binary stability. Unconditional stability does not remove accuracy requirements.

For an advection equation example, the upwind finite difference scheme for $u_t+c{u}_x=0$, $c>0$, has factor $G=1-\nu +\nu e^{-i\theta }$, with $\nu =ck/h$. Sinceit is a contraction in the periodic discrete L2 norm exactly for $0\le \nu \le 1$. In contrast, Forward Euler method with centered advection has factor $1-i\nu \sin \theta$. Its modulus exceeds one at nonzero active frequencies, and repeated amplification makes it unstable under fixed nonzero Courant refinement. A much smaller scaling $k=O(h^2)$ can bound its finite-time growth, demonstrating again that exact contraction and the most general stability estimate are different claims.

The major limitation is the possible presence of a nonnormal matrix. For example,Both eigenvalues are negative, yet at $t=1$ the second unit vector is amplified by at least $1/(eh)$. This proves that negative spectra do not imply uniform semidiscrete stability. Ill-conditioned eigenvectors, Jordan normal form and transient amplification can invalidate a spectral-only argument. Direct energy estimates or estimates for the resolvent can supply the missing norm control.

Boundary conditions are another limitation. A periodic symbol proves a periodic or whole-line result, not an arbitrary initial-boundary-value problem. Boundary closure of a difference scheme can introduce growing modes or mesh-dependent amplification. A spatial operator with variable coefficients can still be studied through its assembled matrix, but then whether it is a normal matrix, its condition number and the physical inner product must be checked rather than borrowed from a Fourier stability analysis. For the finite element method the natural mass matrix inner product often turns a generalized eigenvalue problem into a self-adjoint one.

For time-dependent or split updates, separate spectra are insufficient to control products. For instance, $G_1=\left(\begin{array}{cc}0& K\\ 0& 0\end{array}\right)$ and $G_2=\left(\begin{array}{cc}0& 0\\ K& 0\end{array}\right)$ each have only zero eigenvalues, while $G_2{G}_1$ has eigenvalue $K^2$. For $K>1$, alternating the two is unstable. A common contractive norm would control the products, but their individual eigenvalues do not.

Finally, stability and consistency of a numerical method have different roles. The Lax equivalence theorem states that, for a well-posed linear initial-value problem and a consistent finite difference method on compatible grid spaces, stability on every fixed time interval is equivalent to convergence of a numerical method as the admissible meshes refine. Bounded grid representation and the correct initial and boundary setup are part of that framework. An eigenvalue estimate can establish its stability hypothesis in the favorable settings described above; it neither proves consistency of a numerical method nor bypasses its uniformity requirements. The method is therefore a powerful modal tool, provided its spectral calculation is connected to a genuine norm estimate.

For a normal mode write $x_j=A_j{e}^{i\omega t}$. The generalized eigenvalue problem isThe three independent eigenvectors and corresponding squared frequencies areThe last two have zero mass-weighted displacement and are orthogonal in the mass inner product. The lowest frequency is zero: it represents a rigid translation with both springs unchanged. Its general motion includes uniform centre-of-mass velocity, not just a constant displacement. Superpositions of the two oscillatory normal modes and this translation give the general solution.

This is a generalized eigenvalue problem. A nonzero vector lies in the kernel of $A-\lambda B$ precisely when its determinant vanishes:HenceThe corresponding eigenvectors can be taken as $(1,-1{)}^T$ and $(1,1{)}^T$, respectively, which also checks that both values give nontrivial solutions.

The kinetic energy is $K=\frac{1}{2}m(3{\dot{x}}^2+3{\dot{y}}^2+4{\dot{z}}^2)$. Differentiating the given potential and applying Newton's second law givesThese are the normal modes of a symmetric three-mass string, with mass matrix $M=m\mathrm{diag}(3,3,4)$ and stiffness matrix $m{\Omega }^2\left(\begin{array}{ccc}2& 0& -1\\ 0& 2& -1\\ -1& -1& 2\end{array}\right)$. The coordinate order here is $(x,y,z)$, although the spatial order along the string is $x,z,y$.

Reflection symmetry splits the generalized eigenvalue problem into an antisymmetric mode $x=-y$, $z=0$, with ${\omega }^2=2{\Omega }^2/3$, and symmetric modes $x=y=q$. Writing $\mu ={\omega }^2/{\Omega }^2$, these obey $z=(2-3\mu )q$ and $(2-4\mu )z=2q$. Thus $(2-4\mu )(2-3\mu )-2=2(6\mu -1)(\mu -1)=0$. The three results areThe displacement vectors are mutually orthogonal under the mass-weighted inner product $u^{T}Mv$, as required for these conservative normal modes.