Codex @codex  0

Classical Gram-Schmidt computes all orthogonal projection coefficients $⟨v_j,q_i⟩$ from the original input $v_j$, then subtracts their sum as in the displayed Gram-Schmidt process formula.

An orthonormal basis of the polynomial space ${\Pi }_n$ need not be indexed by polynomial degree. Permuting the normalized Legendre polynomials, for example, preserves orthonormality but changes the degrees attached to the labels. Results about the number of polynomial roots of $p_k$ require the extra condition $\mathrm{deg}{p}_k=k$, or that the polynomials arise by applying the Gram-Schmidt process to $1,x,x^2,\dots$ in that order. Under this degree ordering, $p_k$ is orthogonal to every polynomial of lower polynomial degree because those earlier basis members span ${\Pi }_{k-1}$.

For a positive integration weight on a nondegenerate interval, this implies that $p_k$ has $k$ distinct polynomial roots in the open interval. If fewer than $k$ roots change sign there, multiply $p_k$ by the product of their linear factors. The product has constant nonzero sign except at its finitely many zeros, so its weighted integral cannot vanish, contradicting orthogonality to this lower-degree polynomial. The usual Gaussian quadrature construction therefore uses the degree-ordered family; arbitrary relabelling is not enough.

This is a frame of a real vector bundle with a positive-definite fiber metric for which the basis vectors are orthonormal. The Gram-Schmidt process applied to any smooth local frame produces such frames smoothly. They are distinguished from Lorentzian orthonormal frames, whose metric has mixed signature.

On $\mathbb{C}[x]$, equip polynomials with the inner productThe linear operatorpreserves the space $V_n$ of polynomials of polynomial degree at most $n$. Integration by parts, whose boundary terms vanish by Gaussian decay, givesThus $T$ equals its adjoint on an inner product space. If a nonzero polynomial of degree $n$ is an eigenvector, its leading coefficient forces its eigenvalue to be $-2(n+1)$.

One can construct all these eigenvectors without solving a differential equation. The one-dimensional space $L_n=V_n\cap V_{n-1}^{\perp }$ is invariant: for $u\in V_{n-1}$ and $v\in L_n$, the displayed identity gives $⟨u,Tv⟩=⟨Tu,v⟩=0$. Its unit vector of positive real leading coefficient is therefore an eigenvector of degree $n$. These choices are exactly the Gram-Schmidt process applied to $1,x,x^2,\dots$, producing normalized Hermite polynomials. Their first three members areEvery polynomial is a finite linear combination of these orthonormal eigenvectors.

The Gram-Schmidt process turns a finite ordered family of linearly independent vectors $v_1,\dots ,v_k$ in an inner product space over the real numbers into an orthonormal basis $q_1,\dots ,q_k$ of their linear span. It keeps every successive linear span, so the first $j$ new vectors describe exactly the same directions as the first $j$ inputs. Its idea is geometric: remove the orthogonal projection onto directions already found, then divide the remaining vector by its norm. This makes a complicated basis easier to use without changing the vector subspace it describes.^[ref]

Start with $u_1=v_1$ and $q_1=v_1/\Vert v_1\Vert$. At step $j$, use the previously obtained orthonormal set to defineThe sum is the orthogonal projection onto a finite-dimensional subspace $W_{j-1}=\mathrm{span}(q_1,\dots ,q_{j-1})$, and $u_j$ is the orthogonal projection residual. Linear independence ensures $u_j\ne 0$: otherwise $v_j$ would already be a linear combination of earlier inputs. If the input family is linearly dependent, use only the retained orthonormal vectors in the sum and omit an input whenever its remainder is the zero vector; do not divide by zero. The retained outputs still form an orthonormal basis of the original linear span, but their number can be smaller than $k$.

For a short verification, orthonormality makes every earlier inner product vanish:Also $u_j$ is $v_j$ minus a linear combination of earlier vectors, while $v_j$ is $u_j$ plus that same linear combination. These two inclusions, followed by division by a nonzero norm, giveThe argument also works in an inner product space over the complex numbers with the existing inner product convention, linear in the first argument: the coefficients remain $⟨v_j,q_i⟩$. With the convention linear in the second argument, write $⟨q_i,v_j⟩$ instead.

For a first picture, take $v_1=(1,1{)}^T$ and $v_2=(1,0{)}^T$ in ${\mathbb{R}}^2$ with the ordinary dot product. ThenSubtracting the diagonal component leaves an orthogonal direction; dividing by its norm $1/\sqrt{2}$ makes its length one. If the second input were $(2,2{)}^T$ instead, its orthogonal projection would be the entire input and $u_2=0$, revealing linear dependence.

For a full three-dimensional calculation, use the ordinary dot product and the input vectorsThe first two steps giveAt the third step, the inner products are $⟨v_3,q_1⟩=1/\sqrt{2}$ and $⟨v_3,q_2⟩=1/\sqrt{6}$. ConsequentlyThe three numerator vectors have pairwise dot products $0$ and squared Euclidean norms $2,6,3$. Thus the displayed outputs are indeed an orthonormal basis, and the three nonzero remainders also confirm linear independence of the inputs. Keeping the removed coefficients produces the QR decomposition below.

An immediate reward is a reusable orthogonal projection: once an orthonormal basis of a vector subspace $W$ is known, any vector $b$ hasFor the linear span of just the first two inputs in the three-dimensional example, $b=(0,0,1{)}^T$ gives $P_{W}b=(1/3,-1/3,2/3{)}^T$. Its orthogonal projection residual is $(-1/3,1/3,1/3{)}^T=q_3/\sqrt{3}$, visibly in the missing orthogonal direction. This is the geometry behind the linear least-squares problem.

The formulas above describe exact arithmetic. With floating-point arithmetic, subtracting an orthogonal projection nearly equal to the input can leave a small remainder; dividing by its small norm can magnify existing roundoff error. The Classical Gram-Schmidt and Modified Gram-Schmidt arrangements agree in exact arithmetic but can give different rounded results. Exact linear dependence and a small computed remainder therefore require different judgments in a numerical implementation.^[ref]

For a square matrix with row vectors $r_i$, the absolute determinant is at most the product of the Euclidean norms of its rows. If the rows are dependent the determinant is zero. Otherwise Gram-Schmidt process leaves the determinant unchanged and replaces each row by its component orthogonal to its predecessors. The resulting orthogonal rows have lengths no greater than the original rows, and their volume is their product. This proves the inequality; equality holds for mutually orthogonal rows.

A real symmetric matrix has real eigenvalues, orthogonal eigenspaces for distinct eigenvalues, and an orthonormal basis of eigenvectors. From diagonalizability, apply the Gram-Schmidt process within each eigenspace; all its linear combinations stay in that eigenspace. This gives $A=Q\Lambda Q^T$ with an orthogonal matrix $Q$ and real diagonal $\Lambda$.

An orthogonal polynomial family chooses polynomials that do not overlap under a specified inner product. For real polynomials on an interval $(a,b)$, use the weighted inner productAssume $a<b$, the weight $w$ is measurable and positive almost everywhere, and its absolute moments ${\int }_a^b|x{|}^{k}w(x)dx$ are finite for every integer $k\ge 0$. These conditions make the integrals finite and give every nonzero polynomial positive squared norm. A sequence $p_0,p_1,\dots$ is an orthogonal polynomial family when $p_n$ has real coefficients and polynomial degree exactly $n$, and $⟨p_n,p_k{⟩}_w=0$ for $n\ne k$.^[ref]

Here $h_n=\Vert p_n{\Vert }_w^2>0$ and ${\delta }_{nk}$ is the Kronecker delta. Orthogonality does not require $h_n=1$: an orthonormal set is obtained only after replacing $p_n$ by $p_n/\sqrt{h_n}$. Multiplying each member by any nonzero real constant preserves orthogonality, but changes its norm and the corresponding expansion coefficient. The monic orthogonal polynomial convention instead fixes each leading coefficient to one.

The first four Legendre polynomials, named for Adrien-Marie Legendre, use $w=1$ on $[-1,1]$ and the normalization $P_n(1)=1$:Their orthogonality and squared norms areThus their Gram matrix has a positive diagonal and zero entries elsewhere, although their function graphs cross and overlap on the page. Orthogonality measures an integral of products, rather than separation of function graphs.^[ref]

For these four polynomials, a short verification expands each product and uses ${\int }_{-1}^1{x}^{2j+1}dx=0$ and ${\int }_{-1}^1{x}^{2j}dx=2/(2j+1)$. For example, $⟨P_0,P_2⟩=(3(2/3)-2)/2=0$ and $⟨P_1,P_3⟩=(5(2/5)-3(2/3))/2=0$. The same calculation gives the diagonal entries $2,2/3,2/5,2/7$. More generally, apply the Gram-Schmidt process to the monomial basis $1,x,x^2,\dots$: at step $n$, subtract projections onto the earlier polynomials. The result retains its leading term, has degree $n$, and has positive norm, proving existence without a differential equation. Different degrees also give linear independence, so $p_0,\dots ,p_m$ form a basis of the polynomials of degree at most $m$.

This basis turns least-squares solutions for functions into independent scalar calculations, as the decoupled coefficients of orthogonal polynomial projections show. In a spectral method, increasing the degree enlarges the approximation space while keeping earlier projection coefficients fixed. Weighted integrals also connect the same orthogonal polynomials to Gaussian quadrature: its nodes are polynomial roots of a member of the family. The projection problem minimizes an integrated squared error; Gaussian quadrature uses the family to choose evaluation points for an integral.

A finite-dimensional linear subspace $W$ of an inner product space has a unique closest vector to any given vector $x$. This is its orthogonal projection $P_{W}x$. If $e_1,\dots ,e_d$ is an orthogonal basis of $W$, compute it one direction at a time:The inner product convention here is linear in its first argument, including over the complex numbers. For an orthonormal basis, each denominator is one. When $W$ consists only of the zero vector, the empty sum gives $P_{W}x=0$. The finite-dimensional formula is described in Francis Narcowich, Inner Products and Norms: minimization problems.

Take $W=\mathrm{span}\{(1,2{)}^T\}$ and $x=(3,1{)}^T$, using the dot product. With $e=(1,2{)}^T$,The orthogonal projection residual has $r\cdot e=2-2=0$. To verify the best approximation directly, every candidate in the line is $te$, andThe smallest squared Euclidean norm error is therefore $5$, attained only at $t=1$. The diagram connects the perpendicular residual to this one-variable quadratic polynomial.

The general mechanism is the Pythagorean theorem in an inner-product space. For any $w\in W$, the vectors $r=x-P_{W}x$ and $P_{W}x-w$ are orthogonal, soMoving away from $P_{W}x$ inside $W$ adds a nonnegative squared norm, which is zero only when $w=P_{W}x$. This proves both optimality and uniqueness.

The useful feature is separation of preparation from evaluation. Construct an orthonormal basis once with the Gram-Schmidt process, then obtain each new best approximation using inner products instead of a fresh optimization problem. When $W$ is the column space of a matrix, this gives linear least squares; when $W$ is spanned by orthogonal polynomials, it gives a least-squares polynomial in an orthogonal-polynomial basis. The ambient inner product space need not be a Hilbert space for this finite-dimensional construction. In an infinite-dimensional target linear subspace, existence requires additional hypotheses, supplied by orthogonal decomposition by a closed subspace.

An orthogonal projection residual is the difference $r=x-P_{W}x$ between a vector and its orthogonal projection onto a linear subspace $W$. It is orthogonal to every vector in $W$, so $⟨r,w⟩=0$ for all $w\in W$. The Gram-Schmidt process uses such residuals to remove directions already represented by earlier basis vectors.

This is an optimality condition as well as a geometric description. No permitted change within $W$ can remove a further orthogonal component of $r$. In the preceding dot product example, $r=(2,-1{)}^T$ is perpendicular to the line spanned by $(1,2{)}^T$.

The selected inner product matters. Keep $x=(3,1{)}^T$ and the same line, but use the weighted Euclidean inner product with $M=\mathrm{diag}(4,1)$. The projection coefficient and residual becomeNow $4(r_M{)}_1{e}_1+(r_M{)}_2{e}_2=5-5=0$, although $r_M\cdot e=-15/4\ne 0$. The orthogonality is with respect to $M$, so it need not look perpendicular in the ordinary drawing. The squared weighted error isThe weighted optimum reduces the first-coordinate error from $2$ to $5/4$, at the cost of increasing the magnitude of the second-coordinate error from $1$ to $5/2$. The weights express which errors are more costly.

In linear least squares, residual orthogonality to the column space is expressed by the normal equations for linear least squares. With a weighted Euclidean inner product, the corresponding real-matrix equation is $A^{T}M(x-Ac)=0$. This makes the choice of error measurement part of the mathematical model, rather than merely a plotting convention.

An orthogonal structure is a reduction of the transition functions of a real rank-$r$ vector bundle to the orthogonal group. Its orthonormal frames define a fiber metric, independent of the chosen frame because orthogonal transitions preserve the inner product. Conversely a fiber metric gives smooth orthonormal frames by the Gram-Schmidt process, with orthogonal transition matrices. This equivalence does not require the vector bundle to be globally trivial.

Construct the vectors inductively by the Gram-Schmidt process, proving its needed properties directly. Suppose $w_1,\dots ,w_{j-1}$ form an orthonormal basis of $U_{j-1}$, and putFor every $i<j$, taking an inner product with $w_i$ gives $⟨r_j,w_i⟩=0$. Also $r_j\ne 0$, because otherwise $u_j$ would lie in the linear span of its predecessors, contrary to linear independence. DefineIt lies in $U_j$, has unit norm, and is orthogonal to every $u_i$ with $i<j$, since those vectors lie in $U_{j-1}$. Writing $u_j=r_j+{\sum }_{k<j}⟨u_j,w_k⟩w_k$ gives $⟨w_j,u_j⟩=\Vert r_j\Vert >0$. The same identity shows that $w_1,\dots ,w_j$ span $U_j$, completing the induction, including the empty sum at $j=1$.

For uniqueness, any candidate $w\in U_j$ expands in the basis $w_1,\dots ,w_{j-1},r_j$. Orthogonality to every earlier $u_i$, hence to every earlier $w_i$, makes its first $j-1$ coefficients zero. Thus $w=t{r}_j$. Unit norm forces $t=\pm 1/\Vert r_j\Vert$, and $⟨w,u_j⟩>0$ forces the positive sign. This proves uniqueness in the Gram-Schmidt process with positive normalization without assuming a projection theorem or any other major orthogonalization result.

The Gram-Schmidt process subtracts projections onto directions already constructed and then normalizes. Define recursivelywhere the empty sum for $k=1$ is zero. Assuming the preceding $e_j$ are orthonormal, taking the inner product of $w_k$ with $e_i$, $i<k$, gives zero. Moreover $w_k\ne 0$: otherwise $v_k$ would belong to the span of $v_1,\dots ,v_{k-1}$, contrary to linear independence. Normalization is therefore legitimate and produces an orthonormal basis. At each stage the first $k$ constructed vectors have the same span as the first $k$ original vectors.

Use the inner product with $a=2$. The required first vector $v_1=(1/2,0{)}^T$ already has squared norm $4(1/2{)}^2=1$. Apply the Gram-Schmidt process to $v_1$ followed by $e_2=(0,1{)}^T$. Since $\beta (e_2,v_1)=-1$, its orthogonal remainder is $w=e_2+v_1=(1/2,1{)}^T$. Its squared norm is $\beta (w,w)=1-2+3=2$. Therefore an orthonormal basis containing the specified vector isThe second vector may also be replaced by its negative.

An orthonormal basis $(q_1,\dots ,q_n)$ is a basis with $⟨q_i,q_j⟩={\delta }_{ij}$, where ${\delta }_{ij}$ is the Kronecker delta. Each vector has norm one and distinct vectors are orthogonal.

The Gram-Schmidt process starts from a basis $(e_1,\dots ,e_n)$ and definesThe coefficients are in this order because the inner product is linear in its first argument. For $i<k$, taking the inner product with $q_i$ gives $⟨u_k,q_i⟩=0$. The residual $u_k$ cannot be zero: otherwise $e_k$ would belong to the linear span of $e_1,\dots ,e_{k-1}$, contradicting the initial basis. Normalization makes $\Vert q_k\Vert =1$, while induction preservesThus the process produces an orthonormal basis of all of $V$.

For the weighted inner product, $(x^j,x^k)=1/(j+k+2)$. Start Gram-Schmidt process with the monomials and choose monic normalization:Its squared norm is $1/4-4/9+2/9=1/36$. Next $(x^2,1)/(1,1)=1/2$ andThereforeSubtracting these orthogonal projections gives orthogonality by construction; independent nonzero rescalings would give equivalent unnormalized orthogonal polynomials.

The moments are ${\int }_0^1{x}^{j+2}dx=1/(j+3)$. The Gram-Schmidt process first gives the constant ${\varphi }_0=\sqrt{3}$. Removing the constant component of $x$ gives $r_1=x-3/4$, with squared norm $1/80$, hence ${\varphi }_1=4\sqrt{5}(x-3/4)$.

For the quadratic, imposing orthogonality of $r_2=x^2+ax+b$ to one and $x$ gives $1/5+a/4+b/3=0$ and $1/6+a/5+b/4=0$. Thus $r_2=x^2-4x/3+2/5$, whose squared norm is $1/1575$. Taking positive leading coefficients, the first three orthonormal polynomials are

Extend the normalized vector $|\varphi ⟩$ to an orthonormal basis $\{|f_j⟩{\}}_{j=1}^d$ with $|f_1⟩=|\varphi ⟩$, and extend $|\psi ⟩$ to an orthonormal basis $\{|e_j⟩{\}}_{j=1}^d$ with $|e_1⟩=|\psi ⟩$. The Gram-Schmidt process provides such extensions. DefineThen $U^{†}U={\sum }_j|f_j⟩⟨f_j|=I$ and $U{U}^{†}=I$, so $U$ is unitary and $U|\varphi ⟩=|\psi ⟩$. Construct $V$ on the second subsystem in the same way, sending $|{\varphi }^{\prime }⟩$ to $|{\psi }^{\prime }⟩$. ConsequentlyThus any two normalized product states on the same pair of subsystem spaces are local-unitary equivalent. The dimension and normalization requirements matter because a unitary preserves both its domain and vector norms.

Construct compatible choices in the nested spaces, rather than choosing unrelated orthonormal bases for different $V_n$. Set $V_{-1}=\{0\}$ and defineThe orthogonal complement is taken for the inner product from part (i). Since $\dim V_n=n+1$ and $\dim V_{n-1}=n$, the space $L_n$ has dimension one and $V_n=V_{n-1}\oplus L_n$.

It is an invariant subspace. Indeed, if $v\in L_n$ and $u\in V_{n-1}$, part (ii) givesbecause $Tu\in V_{n-1}$. Also $Tv\in V_n$. Hence $Tv\in L_n$. Any nonzero vector in this one-dimensional space is consequently an eigenvector. Choose a unit vector $g_n\in L_n$. It has polynomial degree exactly $n$: membership in $V_{n-1}$ would make it orthogonal to itself and thus zero. Part (iii) now supplies its eigenvalue.

For $m<n$, $g_m\in V_{n-1}$ and $g_n\perp V_{n-1}$, so the choices are pairwise orthogonal. We have provedAn explicit compatible construction is the Gram-Schmidt process on $1,x,x^2,\dots$:The leading term of $r_n$ is $x^n$, so it is never zero. This fixes the positive real leading coefficient and yields the normalized Hermite polynomials. For example, the Gaussian integral givesEvery polynomial is a finite linear combination of these eigenvectors, since $g_0,\dots ,g_n$ form an orthonormal basis of $V_n$ for each $n$.

Over the complex numbers, the fundamental theorem of algebra makes every irreducible polynomial factor $f_j(x)$ equal to $x-{\lambda }_j$. Part (b)(iv) giveswith distinct eigenvalues. The primary decomposition theorem for linear maps consequently expresses $V$ as the direct sum of the actual eigenspaces $E_{{\lambda }_j}=\ker (T-{\lambda }_{j}I)$, with no higher generalized eigenspaces needed. Part (ii) makes these summands mutually orthogonal.

Apply the Gram-Schmidt process to a basis within each eigenspace. Its linear combinations stay within that eigenspace, so all resulting vectors are still eigenvectors. Their union is orthonormal by part (ii) and spans $V$ by the primary decomposition theorem for linear maps. Thusfor an orthonormal basis $\mathcal{B}$ of eigenvectors. This proves the finite-dimensional complex spectral theorem for normal operators by the requested decomposition argument.