= Solution
An orthonormal <multiresolution analysis> is a sequence of closed subspaces $V_j\subset L^2(\mathbb R)$ satisfying $V_j\subset V_{j+1}$, $\overline{\bigcup_jV_j}=L^2(\mathbb R)$, $\bigcap_jV_j=\{0\}$, and $f\in V_j$ iff $f(2\,\cdot)\in V_{j+1}$. A <scaling function> $\phi$ has integer translates forming an <orthonormal basis> of $V_0$, and its scaled translates $\phi_{j,k}(x)=2^{j/2}\phi(2^jx-k)$ span $V_j$. Thus $V_j$ contains information at resolution about $2^{-j}$.
The detail space $W_j=V_{j+1}\cap V_j^\perp$ is generated by the corresponding <wavelet> functions $\psi_{j,k}(x)=2^{j/2}\psi(2^jx-k)$. The orthogonal splitting $V_{j+1}=V_j\oplus W_j$ iterates to
$$
\boxed{V_J=V_{J_0}\oplus\bigoplus_{j=J_0}^{J-1}W_j.}
$$
Accordingly a square-integrable signal has a coarse-and-detail <wavelet series>,
$$
f=\sum_k a_{J_0,k}\phi_{J_0,k}+\sum_{j\ge J_0}\sum_k\theta_{j,k}\psi_{j,k},
$$
and its <wavelet projection> onto $V_J$ keeps the coarse coefficients and details below level $J$. Increasing $J$ adds finer-scale oscillations and localized features. On the real line there are infinitely many translates at each level; for a bounded observation interval, with an appropriate interval or periodic construction, the relevant approximation dimension satisfies $d_J\asymp2^J$. For the <Haar wavelet> on a unit interval this is exactly $2^J$, corresponding to a separate mean on each dyadic subinterval.
This hierarchy gives a concrete meaning to <wavelet approximation complexity>. Coarse resolution uses few coefficients and can miss sharp features; fine resolution uses more coefficients and can fit noise. By <orthogonality>, projection bias is the energy of the omitted details,
$$
\|f-P_Jf\|_2^2=\sum_{j\ge J}\sum_k|\theta_{j,k}|^2.
$$
For an orthonormal sequence model with independent coefficient noise of variance $\sigma^2/n$, estimating the $d_J$ retained coefficients without shrinkage gives
$$
\boxed{\mathbb E\|\widehat f_J-f\|_2^2=\sum_{j\ge J,k}|\theta_{j,k}|^2+\frac{\sigma^2d_J}{n}.}
$$
The first term decreases with resolution while the second increases, making the <bias-variance tradeoff> explicit. Equivalently, a fixed orthogonal regression projection of rank $d_J$ has <effective degrees of freedom> $d_J$ and prediction optimism $2\sigma^2d_J/n$ under independent equal-variance errors.
Wavelets also permit local rather than only global complexity control. Thresholding small estimated detail coefficients, for example by <soft thresholding>, retains strong local structure while suppressing fine-scale noise elsewhere. A sparse retained coefficient set can represent localized features much more economically than keeping every coefficient through a high level. Its locations and scales matter as well as its count. If that set is selected using the same noisy responses, however, selection adds adaptivity: its raw coefficient count is not automatically its covariance-based degrees of freedom. Resolution or threshold tuning should therefore account for prediction error, using an appropriate risk estimate or <cross-validation>, rather than only decreasing training error.
\b[Wavelet complexity is controlled by retained resolutions and coefficients; more detail reduces approximation bias while increasing estimation variance and sensitivity to noise.]
Back to article page
