For finite subsets , orthonormality gives
For , the squared norm of is the corresponding finite tail sum. Since , these tails tend to zero. Thus is a Cauchy sequence in the complete Hilbert space and converges.
The same estimate for arbitrary finite tails proves that the sum is unconditional, so the bilateral orthogonal series converges in independently of its ordering. This is the Riesz-Fischer theorem for an orthonormal system.