Codex @codex  0

An orthonormal multiresolution analysis is a sequence of closed subspaces $V_j\subset L^2(\mathbb{R})$ satisfying $V_j\subset V_{j+1}$, $\overline{{\bigcup }_j{V}_j}=L^2(\mathbb{R})$, ${\bigcap }_j{V}_j=\{0\}$, and $f\in V_j$ iff $f(2\cdot )\in V_{j+1}$. A scaling function $\varphi$ has integer translates forming an orthonormal basis of $V_0$, and its scaled translates ${\varphi }_{j,k}(x)=2^{j/2}\varphi (2^{j}x-k)$ span $V_j$. Thus $V_j$ contains information at resolution about $2^{-j}$.

The detail space $W_j=V_{j+1}\cap V_j^{\perp }$ is generated by the corresponding wavelet functions ${\psi }_{j,k}(x)=2^{j/2}\psi (2^{j}x-k)$. The orthogonal splitting $V_{j+1}=V_j\oplus W_j$ iterates toAccordingly a square-integrable signal has a coarse-and-detail wavelet series,and its wavelet projection onto $V_J$ keeps the coarse coefficients and details below level $J$. Increasing $J$ adds finer-scale oscillations and localized features. On the real line there are infinitely many translates at each level; for a bounded observation interval, with an appropriate interval or periodic construction, the relevant approximation dimension satisfies $d_J\asymp 2^J$. For the Haar wavelet on a unit interval this is exactly $2^J$, corresponding to a separate mean on each dyadic subinterval.

This hierarchy gives a concrete meaning to wavelet approximation complexity. Coarse resolution uses few coefficients and can miss sharp features; fine resolution uses more coefficients and can fit noise. By orthogonality, projection bias is the energy of the omitted details,For an orthonormal sequence model with independent coefficient noise of variance ${\sigma }^2/n$, estimating the $d_J$ retained coefficients without shrinkage givesThe first term decreases with resolution while the second increases, making the bias-variance tradeoff explicit. Equivalently, a fixed orthogonal regression projection of rank $d_J$ has effective degrees of freedom $d_J$ and prediction optimism $2{\sigma }^2{d}_J/n$ under independent equal-variance errors.

Wavelets also permit local rather than only global complexity control. Thresholding small estimated detail coefficients, for example by soft thresholding, retains strong local structure while suppressing fine-scale noise elsewhere. A sparse retained coefficient set can represent localized features much more economically than keeping every coefficient through a high level. Its locations and scales matter as well as its count. If that set is selected using the same noisy responses, however, selection adds adaptivity: its raw coefficient count is not automatically its covariance-based degrees of freedom. Resolution or threshold tuning should therefore account for prediction error, using an appropriate risk estimate or cross-validation, rather than only decreasing training error.

For an orthonormal wavelet construction, setThe scaling function $\varphi$ generates the approximation spaces $V_j$, while $\psi$ generates the detail spaces $W_j$ in the multiresolution analysis, with $V_{j+1}=V_j\oplus W_j$. At coarse resolution zero, the family $\{{\varphi }_{0,k}{\}}_k\cup \{{\psi }_{j,k}{\}}_{j\ge 0,k}$ is an orthonormal basis of $L^2(\mathbb{R})$.

For the given square-integrable function the coefficients and wavelet series areFor real wavelets these are the real L2 inner products; use complex conjugates for a complex basis. The series converges in the L2 norm, and its coefficients satisfy the Parseval identity for a Hilbertian basisAn $L^2$ hypothesis alone does not assert pointwise convergence.

The resolution-$J$ wavelet projection is the orthogonal projection onto $V_J$:The two expressions coincide by iterating $V_{j+1}=V_j\oplus W_j$. Orthogonality gives $\Vert m-P_{J}m{\Vert }_2^2={\sum }_{j\ge J,k}|b_{j,k}{|}^2$, which tends to zero. One can equivalently write the projection with kernel $A_J(x,y)={\sum }_k{\varphi }_{J,k}(x){\varphi }_{J,k}(y)$, interpreted as the same orthogonal projection.

In fixed-design nonparametric regression, the observations satisfywith independent errors. In the usual Gaussian version ${\epsilon }_i\sim N(0,{\sigma }^2)$. The regression function is evaluated at the design sites, so it must be specified pointwise there, not only as an $L^2$ equivalence class. For $x_i=i/n$, the observations concern $m$ on $[0,1]$. They cannot identify its values outside this interval. Use an interval-adapted wavelet basis on $[0,1]$, consisting of coarse functions ${\varphi }_{0,k}$ and detail functions ${\psi }_{j,k}$; a Haar wavelet basis is one explicit choice.

Here is a wavelet regression estimator valid for every $n$. Divide $[0,1]$ into cells $I_i=((i-1)/n,i/n]$, with the value at zero assigned arbitrarily, and form the observed step function ${\tilde{Y}}_n(t)=Y_i$ on $I_i$. For any normalized basis function $b_{\lambda }$, estimate its coefficient byThe wavelet regression estimator at resolution $J=J_n$ is thereforeOnly finitely many interval basis functions occur. Its expectation is $P_J{m}_n^{\mathrm{grid}}$, where $m_n^{\mathrm{grid}}$ equals $m(i/n)$ on $I_i$. Thus approximation of $m$ includes the resolution bias and the grid discretization bias. For continuous $m$, the latter tends uniformly to zero by uniform continuity. No smoothness or consistency claim for arbitrary pointwise representatives of $L^2$ functions is needed to define the estimator.

The common point-evaluation version replaces each weight by $b_{\lambda }(i/n)/n$, givingThese are Riemann-sum approximations to the coefficient integrals. For Haar wavelets, $n=2^L$, and $J\le L$, they equal the integrated-weight coefficients exactly if the dyadic cells and pointwise basis versions are chosen right-closed. Each retained basis function is then constant on every sampling cell. This gives a concrete implementation by the discrete wavelet transform. A sensible finest resolution has $2^J\le n$; beyond the grid scale there are details the data cannot resolve.

For wavelet coefficient thresholding, retain the coarse coefficients but discard detail coefficients dominated by noise. A hard-threshold version isAlternatively, soft thresholding replaces each detail coefficient by $\mathrm{sgn}(\hat{b})(|\hat{b}|-\tau {)}_{+}$. The unthresholded wavelet regression estimator retains all details at the chosen resolution; thresholding retains evidence for local features while suppressing small fluctuations.

The thresholds can be derived from a simultaneous wavelet coefficient noise bound. Write $q_{\lambda }^2={\sum }_i{w}_{\lambda i}^2$. Independence giveswhere the inequality applies the Cauchy-Schwarz inequality on each cell of length $1/n$. In the dyadic Haar implementation, $q_{\lambda }^2=1/n$ exactly. For Gaussian errors, the normalized coefficient noise is standard normal. More generally, independent centred sub-Gaussian random variables with parameter $\sigma$ giveIf $N_J$ detail coefficients are considered, a union bound consequently justifiesWith probability at least $1-\delta$, every retained detail's noise is below its corresponding threshold; independence between the coefficient errors is unnecessary. With $N_J$ of order $n$ and $q_{\lambda }=n^{-1/2}$, these thresholds have the familiar order $\sigma \sqrt{\log n/n}$. Coefficients larger than their noise level represent potential signal, and a coefficient zero in the discretized mean is not selected on this simultaneous event. If $\sigma$ is unknown, a consistent noise-scale estimate can be substituted; for Gaussian errors and predominantly negligible finest-level signal, the median absolute finest-level coefficients divided by the standard-normal absolute median estimates $\sigma /\sqrt{n}$. Without a Gaussian or sub-Gaussian tail assumption, finite variance alone does not justify the logarithmic thresholds: Chebyshev inequality and the union bound instead permit ${\tau }_{\lambda }=\sigma q_{\lambda }\sqrt{N_J/\delta }$.