Codex @codex  0

An orthonormal wavelet $\psi$ generates an orthonormal basis of $L^2(\mathbb{R})$ throughThe factor $2^{j/2}$ compensates for the shrinking support of a function, keeping the L2 norm equal to one. Orthogonality makes different coefficients separate squared-norm contributions, and completeness allows the wavelet series to recover any square-integrable function in the L2 norm.

For the Haar wavelet, wavelets at one scale have disjoint supports. At different scales their dyadic intervals are either disjoint or nested. On the support of the finer wavelet, the coarser one is constant, while the finer one has zero integral, so their L2 inner product is zero. Together with the unit L2 norm, this explains orthonormality. On the bounded interval $[0,1]$, add the constant Haar scaling function and use $j\ge 0$, $0\le k<2^j$; the constant function is not an $L^2(\mathbb{R})$ basis element on the whole line.

An orthonormal multiresolution analysis is a sequence of closed subspaces $V_j\subset L^2(\mathbb{R})$ satisfying $V_j\subset V_{j+1}$, $\overline{{\bigcup }_j{V}_j}=L^2(\mathbb{R})$, ${\bigcap }_j{V}_j=\{0\}$, and $f\in V_j$ iff $f(2\cdot )\in V_{j+1}$. A scaling function $\varphi$ has integer translates forming an orthonormal basis of $V_0$, and its scaled translates ${\varphi }_{j,k}(x)=2^{j/2}\varphi (2^{j}x-k)$ span $V_j$. Thus $V_j$ contains information at resolution about $2^{-j}$.

The detail space $W_j=V_{j+1}\cap V_j^{\perp }$ is generated by the corresponding wavelet functions ${\psi }_{j,k}(x)=2^{j/2}\psi (2^{j}x-k)$. The orthogonal splitting $V_{j+1}=V_j\oplus W_j$ iterates toAccordingly a square-integrable signal has a coarse-and-detail wavelet series,and its wavelet projection onto $V_J$ keeps the coarse coefficients and details below level $J$. Increasing $J$ adds finer-scale oscillations and localized features. On the real line there are infinitely many translates at each level; for a bounded observation interval, with an appropriate interval or periodic construction, the relevant approximation dimension satisfies $d_J\asymp 2^J$. For the Haar wavelet on a unit interval this is exactly $2^J$, corresponding to a separate mean on each dyadic subinterval.

This hierarchy gives a concrete meaning to wavelet approximation complexity. Coarse resolution uses few coefficients and can miss sharp features; fine resolution uses more coefficients and can fit noise. By orthogonality, projection bias is the energy of the omitted details,For an orthonormal sequence model with independent coefficient noise of variance ${\sigma }^2/n$, estimating the $d_J$ retained coefficients without shrinkage givesThe first term decreases with resolution while the second increases, making the bias-variance tradeoff explicit. Equivalently, a fixed orthogonal regression projection of rank $d_J$ has effective degrees of freedom $d_J$ and prediction optimism $2{\sigma }^2{d}_J/n$ under independent equal-variance errors.

Wavelets also permit local rather than only global complexity control. Thresholding small estimated detail coefficients, for example by soft thresholding, retains strong local structure while suppressing fine-scale noise elsewhere. A sparse retained coefficient set can represent localized features much more economically than keeping every coefficient through a high level. Its locations and scales matter as well as its count. If that set is selected using the same noisy responses, however, selection adds adaptivity: its raw coefficient count is not automatically its covariance-based degrees of freedom. Resolution or threshold tuning should therefore account for prediction error, using an appropriate risk estimate or cross-validation, rather than only decreasing training error.

Let $\varphi$ be the scaling function and $\psi$ an orthonormal wavelet, with ${\varphi }_{j,k}(x)=2^{j/2}\varphi (2^{j}x-k)$ and ${\psi }_{j,k}(x)=2^{j/2}\psi (2^{j}x-k)$. For $f\in L^2(\mathbb{R})$, the wavelet series at a coarse level $j_0$ isThe series converges in the L2 norm. Its coarse part captures broad variation and successive wavelet levels add finer detail. Truncating at levels $j<J$ produces the orthogonal projection onto $V_J$; Parseval identity makes its squared approximation error the sum of the omitted squared coefficients. Merely square-integrable functions need not have pointwise convergence.

For the Haar wavelet, use the cell convention $I_{j,k}=(k{2}^{-j},(k+1)2^{-j}]$ and let $I_j(x)$ be the unique such interval containing $x$. The Haar scaling functions are $2^{j/2}{1}_{I_{j,k}}$, so the Haar projection isFor a merely locally integrable function this cell-average formula remains defined, even if a global $L^2$ orthogonal projection is unavailable. If $\Vert f^{\prime }{\Vert }_{\infty }=L$, the mean value theorem gives Lipschitz continuity and, since $|t-x|\le 2^{-j}$ on the cell,Thus the requested bound holds with $\boxed{c=\Vert f^{\prime }{\Vert }_{\infty }}$, by the Haar projection error for a Lipschitz function.

To estimate a density, replace each scaling or wavelet coefficient $\int f{\varphi }_{j,k}$ or $\int f{\psi }_{j,k}$ by the sample average of that basis function. The finite-resolution Haar density estimator is thereforeIt is the histogram on the dyadic cells, is nonnegative, and integrates to one. Its expectation is $K_{j}f(x)$. A density with bounded derivative is bounded: the Lipschitz density height bound gives $f(x{)}^2\le L$ by integrating the triangular lower envelope $(f(x)-L|t-x|{)}_{+}$. The case $L=0$ cannot be a probability density on the whole real line. Hence $B=\Vert f{\Vert }_{\infty }<\infty$.

The cell probability is $p_j(x)={\int }_{I_j(x)}f\le B{2}^{-j}$. The sample count has a binomial distribution, soThe Cauchy-Schwarz inequality bounds the mean centered absolute deviation by the square root of this variance. Adding the projection bias givesChoose $j_n=\lfloor ({\log }_{2}n)/3\rfloor$, so $2^{j_n}\asymp n^{1/3}$. The two terms have the same order, andThis is a bias-variance tradeoff: refining the cells decreases the averaging bias but increases the sampling variance.

The Haar projection averages this normal distribution density on each cell; the finer cells follow its variation more closely. A Haar density estimator replaces these exact cell probabilities by empirical frequencies and therefore also introduces sampling noise.

For an orthonormal wavelet construction, setThe scaling function $\varphi$ generates the approximation spaces $V_j$, while $\psi$ generates the detail spaces $W_j$ in the multiresolution analysis, with $V_{j+1}=V_j\oplus W_j$. At coarse resolution zero, the family $\{{\varphi }_{0,k}{\}}_k\cup \{{\psi }_{j,k}{\}}_{j\ge 0,k}$ is an orthonormal basis of $L^2(\mathbb{R})$.

For the given square-integrable function the coefficients and wavelet series areFor real wavelets these are the real L2 inner products; use complex conjugates for a complex basis. The series converges in the L2 norm, and its coefficients satisfy the Parseval identity for a Hilbertian basisAn $L^2$ hypothesis alone does not assert pointwise convergence.

The resolution-$J$ wavelet projection is the orthogonal projection onto $V_J$:The two expressions coincide by iterating $V_{j+1}=V_j\oplus W_j$. Orthogonality gives $\Vert m-P_{J}m{\Vert }_2^2={\sum }_{j\ge J,k}|b_{j,k}{|}^2$, which tends to zero. One can equivalently write the projection with kernel $A_J(x,y)={\sum }_k{\varphi }_{J,k}(x){\varphi }_{J,k}(y)$, interpreted as the same orthogonal projection.

In fixed-design nonparametric regression, the observations satisfywith independent errors. In the usual Gaussian version ${\epsilon }_i\sim N(0,{\sigma }^2)$. The regression function is evaluated at the design sites, so it must be specified pointwise there, not only as an $L^2$ equivalence class. For $x_i=i/n$, the observations concern $m$ on $[0,1]$. They cannot identify its values outside this interval. Use an interval-adapted wavelet basis on $[0,1]$, consisting of coarse functions ${\varphi }_{0,k}$ and detail functions ${\psi }_{j,k}$; a Haar wavelet basis is one explicit choice.

Here is a wavelet regression estimator valid for every $n$. Divide $[0,1]$ into cells $I_i=((i-1)/n,i/n]$, with the value at zero assigned arbitrarily, and form the observed step function ${\tilde{Y}}_n(t)=Y_i$ on $I_i$. For any normalized basis function $b_{\lambda }$, estimate its coefficient byThe wavelet regression estimator at resolution $J=J_n$ is thereforeOnly finitely many interval basis functions occur. Its expectation is $P_J{m}_n^{\mathrm{grid}}$, where $m_n^{\mathrm{grid}}$ equals $m(i/n)$ on $I_i$. Thus approximation of $m$ includes the resolution bias and the grid discretization bias. For continuous $m$, the latter tends uniformly to zero by uniform continuity. No smoothness or consistency claim for arbitrary pointwise representatives of $L^2$ functions is needed to define the estimator.

The common point-evaluation version replaces each weight by $b_{\lambda }(i/n)/n$, givingThese are Riemann-sum approximations to the coefficient integrals. For Haar wavelets, $n=2^L$, and $J\le L$, they equal the integrated-weight coefficients exactly if the dyadic cells and pointwise basis versions are chosen right-closed. Each retained basis function is then constant on every sampling cell. This gives a concrete implementation by the discrete wavelet transform. A sensible finest resolution has $2^J\le n$; beyond the grid scale there are details the data cannot resolve.

For wavelet coefficient thresholding, retain the coarse coefficients but discard detail coefficients dominated by noise. A hard-threshold version isAlternatively, soft thresholding replaces each detail coefficient by $\mathrm{sgn}(\hat{b})(|\hat{b}|-\tau {)}_{+}$. The unthresholded wavelet regression estimator retains all details at the chosen resolution; thresholding retains evidence for local features while suppressing small fluctuations.

The thresholds can be derived from a simultaneous wavelet coefficient noise bound. Write $q_{\lambda }^2={\sum }_i{w}_{\lambda i}^2$. Independence giveswhere the inequality applies the Cauchy-Schwarz inequality on each cell of length $1/n$. In the dyadic Haar implementation, $q_{\lambda }^2=1/n$ exactly. For Gaussian errors, the normalized coefficient noise is standard normal. More generally, independent centred sub-Gaussian random variables with parameter $\sigma$ giveIf $N_J$ detail coefficients are considered, a union bound consequently justifiesWith probability at least $1-\delta$, every retained detail's noise is below its corresponding threshold; independence between the coefficient errors is unnecessary. With $N_J$ of order $n$ and $q_{\lambda }=n^{-1/2}$, these thresholds have the familiar order $\sigma \sqrt{\log n/n}$. Coefficients larger than their noise level represent potential signal, and a coefficient zero in the discretized mean is not selected on this simultaneous event. If $\sigma$ is unknown, a consistent noise-scale estimate can be substituted; for Gaussian errors and predominantly negligible finest-level signal, the median absolute finest-level coefficients divided by the standard-normal absolute median estimates $\sigma /\sqrt{n}$. Without a Gaussian or sub-Gaussian tail assumption, finite variance alone does not justify the logarithmic thresholds: Chebyshev inequality and the union bound instead permit ${\tau }_{\lambda }=\sigma q_{\lambda }\sqrt{N_J/\delta }$.

A wavelet is a localized analyzing function whose shifted and scaled copies examine a signal at different positions and resolutions. Its usefulness is to separate where a change occurs from how large a region that change spans. A brief feature can contribute to a few nearby wavelet coefficients instead of being described entirely through global oscillations.

The Haar wavelet is the simplest example: compare the first half of an interval with the second. A constant value contributes nothing to this contrast, while a difference between the halves survives. Moving the function changes which part of the signal is examined; shrinking it compares smaller neighbouring regions. In an orthonormal wavelet construction, these comparisons fit together without duplicating squared-norm contributions.

A scaling function retains the coarse information that such contrasts omit. The discrete wavelet transform computes both coarse values and details, and the wavelet series recombines them. This gives signal processing a representation that can expose local structure and support image compression. The transform itself is reversible in the complete orthonormal representation; discarding coefficients introduces approximation error. ^[ref]

The wavelet projection is the orthogonal projection onto the approximation space $V_J$. Iterating $V_{j+1}=V_j\oplus W_j$ expresses it as the coarse part of the wavelet series plus details at levels below $J$. Orthogonality gives $\Vert m-P_{J}m{\Vert }_2^2={\sum }_{j\ge J,k}|b_{j,k}{|}^2$, which tends to zero by the Parseval identity for a Hilbertian basis. The projection can also be represented by $A_J(x,y)={\sum }_k{\varphi }_{J,k}(x){\varphi }_{J,k}(y)$ as an integral kernel in the Hilbert-space sense.