= Orthogonal projection
{title2=$P$}
= Orthogonal projector
{synonym}
The orthogonal projection $P_F$ onto a <closed subspace of a Hilbert space> $F$ sends $x$ to its component in the <orthogonal decomposition>
$$
x=P_Fx+r,\qquad P_Fx\in F,\quad r\in F^\perp.
$$
The residual $r$ is <orthogonal> to every permitted direction. For any $z\in F$, the <Pythagorean theorem> gives
$$
\|x-z\|^2=\|r\|^2+\|P_Fx-z\|^2.
$$
Thus $P_Fx$ uniquely minimizes the distance to $F$. It is a <linear map>, is idempotent, and is <nonexpansive>: $\|P_Fx-P_Fy\|\leq\|x-y\|$; taking $y=0$ gives $\|P_Fx\|\leq\|x\|$.
In $\mathbb R^2$, project $v=(1,3)$ onto the line spanned by $u=(2,1)$. The <inner product> gives
$$
P_Fv=\frac{\langle v,u\rangle}{\langle u,u\rangle}u
=\frac55u=(2,1),\qquad r=(-1,2).
$$
The check $\langle r,u\rangle=-2+2=0$ confirms orthogonality, and $\|v\|^2=10=5+5=\|P_Fv\|^2+\|r\|^2$. Projecting again changes nothing, since the result already lies on the line.
\Image[/orthogonal-projection-geometry.png]
{title=Orthogonal projection gives the unique nearest point on a line and a perpendicular residual}
For a finite <orthonormal basis> $e_1,\ldots,e_m$ of $F$, the same operation is $P_Fx=\sum_j\langle x,e_j\rangle e_j$, using the convention linear in the first argument. Replacing coordinate <vectors> by <functions> gives exactly the same approximation principle. For $f(x)=x^2$ in the real <L2 space> on $[-1,1]$ and $F=\operatorname{span}\{1,x\}$, the directions are <orthogonal>, with squared <norms> $2$ and $2/3$. Their <inner products> with $f$ are $2/3$ and zero, so
$$
P_Ff=\frac{2/3}{2}\cdot1+\frac0{2/3}\cdot x=\frac13.
$$
Direct integration gives the squared error $\|x^2-1/3\|^2=8/45$. For any competing affine <polynomial>, the <orthogonal> residual yields
$$
\int_{-1}^{1}\bigl(x^2-a-bx\bigr)^2\,dx
=\frac8{45}+2\left(a-\frac13\right)^2+\frac23b^2.
$$
Every deviation from $a=1/3$, $b=0$ adds a nonnegative error. The best <function> need not pass through any selected data point: it minimizes an integrated error across the whole <interval>.
\Image[/hilbert-function-approximation.png]
{title=Projecting x squared onto affine functions gives the constant one third and minimum integrated squared error 8/45}
Back to article page
