 Polynomial least-squares approximation

For a positive-definite weighted inner product on a compact interval, a least-squares polynomial is the orthogonal projection of a function onto the finite-dimensional space ${\Pi }_n$ of polynomials of degree at most $n$. In a basis ${\varphi }_0,\dots ,{\varphi }_n$, its coefficients solve the normal equations $Gc=d$, where $G_{ij}=({\varphi }_i,{\varphi }_j)$ and $d_i=(u,{\varphi }_i)$. The Gram matrix is positive definite: $z^{T}Gz=\Vert \sum z_i{\varphi }_i{\Vert }^2>0$ for $z\ne 0$. Thus the coefficients exist and are unique. The residual is orthogonal to every candidate polynomial, giving $\Vert u-q{\Vert }^2=\Vert u-p_n{\Vert }^2+\Vert p_n-q{\Vert }^2$ and proving optimality. An orthogonal basis makes these equations diagonal, as in a least-squares polynomial in an orthogonal-polynomial basis.