Basis Created 2026-09-24 Updated 2026-10-14
A basis is a linearly independent spanning set of vectors in a vector space. Every vector is a unique finite linear combination of basis vectors. Independence supplies uniqueness, and spanning supplies existence. An ordered basis turns the coefficients into a coordinate vector.
For , the matrix with these vectors as columns is
Hence the coordinate vector of is , and recovers the usual coordinates. The determinant is nonzero, so this recovery works for every vector in . A basis need not have perpendicular or unit-length vectors; an orthonormal basis adds those geometric conveniences when an inner product is available.
The point of changing basis is to simplify the operation being studied. Diagonalization uses an eigenbasis to turn a coupled linear map into independent scalings. In a space of polynomials, the basis makes differentiation easy to encode, while orthogonal polynomials can make approximation errors easier to measure. Gilbert Strang's MIT lecture on independence, basis and dimension develops this connection between attainable columns, redundant columns and coordinates.[ref]
Chebyshev polynomial Created 2026-09-24 Updated 2026-10-18
The first-kind Chebyshev polynomial converts a multiple-angle cosine into a polynomial:
The cosine addition formula gives the recurrence relation
This constructs polynomials valid for every real , including outside , where the real-angle description is unavailable. For example,
For , has polynomial degree and leading coefficient . On its uniform norm is one, even as its oscillations become more numerous.[ref][ref]
The polynomial roots of are especially simple:
They correspond to equally spaced angles, but their locations in cluster near and . For , the polynomial roots are . For , the positive polynomial roots are approximately , with their negatives completing the set. The distance from the right endpoint to the nearest polynomial root is ; central neighboring polynomial roots are separated on the larger scale . The small change of cosine near angles zero and explains the clustering.
Figure 1.
Analytical illustration of the eight roots of T8: equally spaced angles on a semicircle project to endpoint-clustered interpolation nodes
. Details.
These polynomial roots supply interpolation nodes for a polynomial interpolation of polynomial degree at most . They differ from the Chebyshev-Lobatto nodes , which include both endpoints. This choice of polynomial roots has nodal polynomial
The monic Chebyshev extremal polynomial minimizes the uniform norm of this product among monic polynomials of polynomial degree on . Consequently, if has continuous derivatives, the usual polynomial interpolation error gives the useful bound
This optimizes the nodal factor in that bound, not the actual error for every function. Endpoint clustering helps control the large endpoint oscillations of equally spaced interpolation associated with Runge's phenomenon. The two node conventions and their geometric construction are discussed in Trefethen's authored text.[ref]
The same cosine substitution for polynomial approximation explains the weighted orthogonality:
The endpoint singularities of the weight are integrable. Thus the Chebyshev polynomials are orthogonal polynomials for this weight, making coefficients in a least-squares polynomial in an orthogonal-polynomial basis separate cleanly. Their polynomial roots also give Chebyshev–Gauss quadrature, connecting polynomial interpolation to Gaussian quadrature.[ref]
Accuracy still depends on the target function. The absolute value function has a corner at zero, so the hypothesis requiring continuous derivatives in the displayed polynomial interpolation error bound fails. Endpoint clustering cannot remove that corner. A jump poses a stronger obstruction: polynomials are continuous functions, so they cannot converge in the uniform norm to a discontinuous function on the whole interval. Chebfun's documented examples show persistent overshoot for such jumps; these are a limitation of global polynomial interpolation, not a universal promise of rapid accuracy from Chebyshev polynomials.[ref]
Claude Brezinski Created 2026-10-18 Updated 2026-11-05
Claude Brezinski works in numerical analysis, particularly extrapolation, Padé approximation, continued fractions, orthogonal polynomials and numerical linear algebra. His research also treats systems of nonlinear equations. These subjects connect different representations of an approximation with ways to accelerate or stabilize a numerical computation.[ref]
Identity source: claude-brezinski.fr/
Decoupled coefficients of orthogonal polynomial projections Created 2026-10-18 Updated 2026-10-20
Let be a real function with , and let be the vector space of real polynomials of degree at most . Its orthogonal projection is the least-squares polynomial in an orthogonal-polynomial basis
Each coefficient depends on just one weighted inner product with and one squared norm. In particular, the coefficients stay the same when the allowed polynomial degree increases. This statement assumes a fixed weight, interval and normalization of the orthogonal polynomials.
To see why the least-squares normal equations decouple, first write a trial polynomial as in any basis of . The residual must be orthogonal to each permitted direction, giving
The Gram matrix usually couples the unknown coefficients. With , it becomes , leaving the separate equations . For the monomial basis on with , by contrast, is zero when is odd and when is even. Distinct even powers, or distinct odd powers, remain coupled.
There is also a proof of the minimum that uses no differentiation. The defining coefficients give for . Thus is orthogonal to all of . For every , the Pythagorean theorem in an inner-product space gives
The second term is nonnegative and vanishes exactly when , proving that the least-squares solution is unique. This is an integrated squared-error optimum; it does not assert a smallest error at every point or a smallest uniform norm.
For , the nested orthogonal projections satisfy the exact residual budget
Indeed, is a linear combination of mutually orthogonal polynomials, and is orthogonal to it. Every newly retained component removes exactly from the squared error. No infinite expansion or completeness assumption is needed for this finite identity. The Legendre least-squares projections of a cubic make every term explicit.
Let be the vector space of real polynomials of degree of a polynomial at most . Choose distinct real sample points and positive real weights . Then
is an inner product. Symmetry and bilinearity follow from linearity of polynomial evaluation. Its value at is nonnegative, and equality forces all . By the root bound for a polynomial, a nonzero polynomial of degree of a polynomial at most cannot vanish at distinct points. Thus the form has positive definiteness.
Equivalently, the polynomial evaluation map on a finite point set identifies with , equipped with a weighted Euclidean inner product. Too few sample points destroy positive definiteness: if the distinct points number at most , the nonzero polynomial lies in and has zero sampled squared norm.
For three equally weighted samples at , an orthonormal basis is
This construction gives a finite sampling version of orthogonal polynomials and permits exact Gram-Schmidt orthogonalization using only finitely many evaluations.
Gegenbauer polynomial Created 2026-09-29 Updated 2026-11-05
For , the Gegenbauer polynomials are normalized by the generating function
near . This fixes the scale left undetermined by the Gegenbauer differential equation. The first examples are , and . They are orthogonal polynomials on for the weight . At they are the Legendre polynomials; at they give the Chebyshev polynomial of the second kind family.[ref]
At distinct interpolation nodes , prescribed values and first derivatives determine exactly one Hermite interpolating polynomial of polynomial degree at most . The homogeneous data force a double root of a polynomial at every interpolation node. Hence the kernel of a linear map given by these conditions is zero: a nonzero polynomial with those double roots of a polynomial would have polynomial degree at least . Equal finite dimensions of a vector space then give existence as well as uniqueness. This interpolation polynomial is distinct from the Hermite polynomials forming a classical orthogonal polynomial family.
Take the polynomial on with unit weight. Using the four Legendre polynomials above, direct integration gives
For example, : the omitted odd terms have zero integral. Dividing by gives . Substituting all four coefficients proves the exact identity
The successive orthogonal projections onto degrees at most are therefore
Here is the squared L2 norm, without division by the interval length. Each row retains the previous Legendre polynomial coefficients and appends one new term. The ordinary power coefficients can change: adding changes the constant from to , and adding changes the linear coefficient from to . Independence belongs to the chosen orthogonal polynomial coordinates.
Figure 1.
Nested Legendre least-squares projections of a cubic and the exact squared residual contributed by each omitted mode
. Details.
The full squared L2 norm is , equal to . The residual budgets are the omitted terms:
For a direct check independent of summing the budgets, , and integrating its square gives . This residual is orthogonal to , exactly the conditions for the best quadratic least-squares solution. An orthogonal projection need not be a polynomial interpolant: for example, while . The approximation allocates error across the interval according to the chosen weighted inner product.
The Laguerre polynomial normalized by has coefficients . By the Leibniz rule, this is equivalent to Rodrigues' formula, . For any polynomial of degree below , applications of integration by parts give . All endpoint terms vanish: derivatives of of order below vanish at zero and decay at infinity. Since , the same calculation gives
Thus these orthogonal polynomials are already an orthonormal set for the weighted inner product with weight .
Orthogonal polynomial Created 2026-09-24 Updated 2026-10-18
An orthogonal polynomial family chooses polynomials that do not overlap under a specified inner product. For real polynomials on an interval , use the weighted inner product
Assume , the weight is measurable and positive almost everywhere, and its absolute moments are finite for every integer . These conditions make the integrals finite and give every nonzero polynomial positive squared norm. A sequence is an orthogonal polynomial family when has real coefficients and polynomial degree exactly , and for .[ref]
Here and is the Kronecker delta. Orthogonality does not require : an orthonormal set is obtained only after replacing by . Multiplying each member by any nonzero real constant preserves orthogonality, but changes its norm and the corresponding expansion coefficient. The monic orthogonal polynomial convention instead fixes each leading coefficient to one.
The first four Legendre polynomials, named for Adrien-Marie Legendre, use on and the normalization :
Their orthogonality and squared norms are
Thus their Gram matrix has a positive diagonal and zero entries elsewhere, although their function graphs cross and overlap on the page. Orthogonality measures an integral of products, rather than separation of function graphs.[ref]
Figure 1.
The first four Legendre polynomials and their exact integral Gram matrix: orthogonal, with unequal squared norms
. Details.
For these four polynomials, a short verification expands each product and uses and . For example, and . The same calculation gives the diagonal entries . More generally, apply the Gram-Schmidt process to the monomial basis : at step , subtract projections onto the earlier polynomials. The result retains its leading term, has degree , and has positive norm, proving existence without a differential equation. Different degrees also give linear independence, so form a basis of the polynomials of degree at most .
This basis turns least-squares solutions for functions into independent scalar calculations, as the decoupled coefficients of orthogonal polynomial projections show. In a spectral method, increasing the degree enlarges the approximation space while keeping earlier projection coefficients fixed. Weighted integrals also connect the same orthogonal polynomials to Gaussian quadrature: its nodes are polynomial roots of a member of the family. The projection problem minimizes an integrated squared error; Gaussian quadrature uses the family to choose evaluation points for an integral.
At equally replicated quantitative treatment levels, evaluations of orthogonal polynomials generate pairwise orthogonal treatment contrasts, using the replication-weighted inner product. Five equally spaced levels give four contrast directions representing degrees one through four, besides the constant direction. This decomposes treatment variation without assuming the response has low polynomial degree.
Orthogonal projection onto a finite-dimensional subspace Created 2026-09-24 Updated 2026-10-14
A finite-dimensional linear subspace of an inner product space has a unique closest vector to any given vector . This is its orthogonal projection . If is an orthogonal basis of , compute it one direction at a time:
The inner product convention here is linear in its first argument, including over the complex numbers. For an orthonormal basis, each denominator is one. When consists only of the zero vector, the empty sum gives . The finite-dimensional formula is described in Francis Narcowich, Inner Products and Norms: minimization problems.
Take and , using the dot product. With ,
The orthogonal projection residual has . To verify the best approximation directly, every candidate in the line is , and
The smallest squared Euclidean norm error is therefore , attained only at . The diagram connects the perpendicular residual to this one-variable quadratic polynomial.
Figure 1.
The perpendicular residual identifies the closest point on a line and the unique minimum of the squared error
. Details.
The general mechanism is the Pythagorean theorem in an inner-product space. For any , the vectors and are orthogonal, so
Moving away from inside adds a nonnegative squared norm, which is zero only when . This proves both optimality and uniqueness.
The useful feature is separation of preparation from evaluation. Construct an orthonormal basis once with the Gram-Schmidt process, then obtain each new best approximation using inner products instead of a fresh optimization problem. When is the column space of a matrix, this gives linear least squares; when is spanned by orthogonal polynomials, it gives a least-squares polynomial in an orthogonal-polynomial basis. The ambient inner product space need not be a Hilbert space for this finite-dimensional construction. In an infinite-dimensional target linear subspace, existence requires additional hypotheses, supplied by orthogonal decomposition by a closed subspace.
Orthonormal polynomial 2026-10-31
An orthonormal family of orthogonal polynomials also gives each member unit norm in the chosen weighted inner product. Its pairwise inner products are the Kronecker delta. Dividing every nonzero orthogonal polynomial by its norm produces such a family; the sign or complex phase still requires a convention, such as a positive real leading coefficient.
Assume a positive even weight on a symmetric domain has finite polynomial moments. Subtracting the orthogonal projection onto a finite-dimensional subspace of onto the lower-degree polynomials gives the unique monic orthogonal polynomial. Reflection produces another such monic polynomial, , so uniqueness gives the parity identity. Every coefficient with the opposite parity to its degree therefore vanishes.
For a positive even weight on a symmetric interval, any orthogonal polynomial of degree has the parity of . Induct on . The reflected polynomial has the same leading coefficient. By substitution in the weighted inner product and the inductive parity of all lower polynomials, it is orthogonal to them. Hence has lower degree and is orthogonal to the entire space containing it. Its squared norm is zero, forcing . The base case is a constant. This proof needs neither monic nor unit-norm normalization; it only compares polynomials with the same leading coefficient.
Put in the weighted inner product. Since ,
The last equality follows by the product-to-sum trigonometric identity and integration. Thus these are nonzero mutually orthogonal polynomials. The first are linearly independent, and the vector space has dimension ; therefore .
For any , the function of the real parameter
has a minimum at . Its derivative there vanishes, proving . This is precisely the defining orthogonality of an orthogonal projection, or a least-squares polynomial in an orthogonal-polynomial basis. Conversely, this condition and the Pythagorean theorem for inner product spaces give , proving the minimum and its uniqueness.
Expand in the basis already established and take the inner product with . Orthogonality leaves , so
In particular and for . These are the corresponding Chebyshev series coefficients with the constant coefficient convention used in this question.
Use the standard convention that the orthogonal polynomials are nonzero. Their exact degrees then follow from the stated assumptions by induction: is a nonzero constant, and if span , a nonzero orthogonal to all of them cannot lie in .
Let be the distinct sign-changing roots of in , and put . Then has constant sign between and across all its roots, and is not identically zero. Because in the interior, is nonzero. If , orthogonality to would make this integral zero. Thus .
A degree- polynomial cannot have more than roots counted with multiplicity, so all roots are real, simple, and strictly inside . This is the standard zeros of orthogonal polynomials argument, with the positive weight ensuring strict noncancellation.
The displayed recurrence requires monic normalization, which is not explicitly stated in the preamble. Arbitrary independent rescaling of orthogonal polynomials preserves orthogonality but changes the recurrence coefficients. Choose their monic representatives for this and the following calculations.
Expand in the orthogonal polynomial basis:
The leading coefficient of is one. For , , because the second factor has degree below . Thus only remain. Inner products give
The last equality uses . Rearranging yields exactly the three-term recurrence for monic orthogonal polynomials
Let be the monic node polynomial. Suppose first that the quadrature is exact through degree . For any polynomial of degree at most , has degree at most , while every sampled value is zero. Exactness gives
Thus is an orthogonal polynomial of degree , proving necessity in the orthogonal-node criterion for Gaussian quadrature.
Conversely, suppose the nodes are the zeros of such a polynomial. Rescale it to be monic without changing its zeros or orthogonality. For any of degree at most , polynomial division gives
Orthogonality makes ; the quadrature of is zero because . The assumed degree- exactness applies to , so
Hence degree- exactness is equivalent to orthogonal nodes. The given distinct-zero property ensures the node values are independent interpolation data; the degree- exactness fixes the corresponding interpolatory weights. This is the construction of Gaussian quadrature.
Use the weighted inner product . A monic quadratic orthogonal polynomial , orthogonal to 1 and , satisfies
Thus , , and its two zeros are and . Matching the first two weighted moments gives
so and . The two-node Gaussian quadrature with linear weight is therefore
For completeness, a polynomial of degree at most three can be divided as , where both and have degree at most one. Orthogonality makes the weighted integral of zero, and this term vanishes at both nodes. The two moment equations integrate exactly. Thus the Gaussian quadrature is exact for every cubic polynomial. No two-node rule can integrate every quartic: the square of its nodal polynomial has quadrature value zero and strictly positive weighted integral.