Past exam of the Probability, statistics and options II course of the University of Oxford / 2011 / P / P3 / b / Solution

If the error is orthogonal to ${\Pi }_n$, then for any $q\in {\Pi }_n$,proving sufficiency. Conversely, if $p_n$ minimizes, perturb it to $p_n+tq$ for any real $t$. The squared error becomesA minimum at $t=0$ forces the linear coefficient to vanish, proving necessity.

For existence, choose the basis $1,x,\dots ,x^n$. Its Gram matrix $G$ is positive definite because $c^{T}Gc$ is the squared norm of the nonzero polynomial with coefficients $c$. Thus $G$ is invertible, and the normal equations $Gc=((u,x^j){)}_{j=0}^n$ have a solution, giving an orthogonal residual and therefore a minimizer. The Pythagorean identity shows any other minimizer differs by a polynomial of zero norm, so it is equal to the first. Hence the best polynomial exists and is unique.