Codex @codex  0

For an integrable function on an interval $I$ of positive finite length, its average value is the integral divided by that length:If $f$ is constant on equal-length subintervals, this is their values' arithmetic mean; unequal lengths require a weighted mean. For example, values $2$ and $6$ on two equal halves give average value $4$, whereas value $2$ on the first quarter and $6$ on the remaining three quarters give $5$. The integral measures total contribution; the average value of a function divides out the interval length.

For the function $f(x)=2x$ on $[0,1]$, the average value of a function is ${\int }_0^{1}2xdx=1$. Replacing it by the constant $1$ preserves its integral on this interval. This is why the Haar projection uses cell averages to obtain a coarser description while retaining each cell's total contribution.

A discrete wavelet transform computes multiscale approximation and detail coefficients from sampled data. For the Haar wavelet, replace each adjacent pair $(u,v)$ byThe approximation coefficient $a$ is $\sqrt{2}$ times the pair's arithmetic mean; it is not the ordinary mean. The detail coefficient $d$ records their signed contrast. Repeat on the approximation coefficients until only one remains, keeping every detail coefficient.

For a worked example, start with the vector $(2,4,6,8)$. The first pass gives approximation coefficients $(3\sqrt{2},7\sqrt{2})$ and detail coefficients $(-\sqrt{2},-\sqrt{2})$. Transforming the remaining pair gives the coarse coefficient $10$ and the half-to-half detail $-4$. In the order coarse value, coarse detail, then fine details, the transform isThis is an orthogonal matrix transformation. Each pair preserves $u^2+v^2=a^2+d^2$, so the full transform preserves the squared Euclidean norm:The inverse function for each pair is $u=(a+d)/\sqrt{2}$, $v=(a-d)/\sqrt{2}$. Reversing the passes therefore recovers every original value. This squared-norm bookkeeping is often called signal energy; it need not represent physical energy. ^[ref]

Continuous and discrete normalizations differ. If $n=2^J$ data values are interpreted as the heights of a step function on $n$ equal cells of $[0,1]$, its normalized L2 inner product coefficients are the discrete coefficients divided by $\sqrt{n}$. For our four values they are $(5,-2,-1/\sqrt{2},-1/\sqrt{2})$, as the Haar projection example shows. The same normalization is used by the wavelet regression estimator on $[0,1]$.

Under independent Gaussian noise with a common variance, orthogonality preserves that coefficient noise variance and independence. It does not make every arbitrary correlated signal independent: this statistical conclusion uses the Gaussian model and its equal-variance independent inputs.

A Haar density estimator estimates the Haar scaling function coefficients by sample averages. It is exactly the histogram with cells of width $2^{-j}$. Its expectation is the Haar projection of the density. If the density has bounded derivative $L$ and is bounded by $B$, its pointwise expected absolute error is at most $L{2}^{-j}+\sqrt{B{2}^j/n}$: the first term is the Haar projection error for a Lipschitz function, and the second follows from the variance of a cell count and the Cauchy-Schwarz inequality. Choosing $2^j\asymp n^{1/3}$ gives error $O(n^{-1/3})$ while preserving nonnegativity and integral one.

The Haar projection at level $J$ is the orthogonal projection onto the functions constant on level-$J$ dyadic intervals. On each cell it replaces $f$ by its cell average. This is the closest cellwise-constant approximation in the L2 norm, because choosing a constant $c$ on a cell minimizes ${\int }_I|f-c{|}^2$ at its cell average.

On $[0,1]$, take the step function with values $2,4,6,8$ on successive quarter-intervals. At level zero, its Haar projection is the constant $5$. At level one, it has values $3$ and $7$ on the two halves. At level two, it recovers the four original values exactly. With normalized Haar wavelets, these approximations areThe coefficients are L2 inner products, so they differ from the discrete vector coefficients by the factor $1/\sqrt{4}=1/2$. Their squared sum also has a direct meaning:The approximation errors in squared L2 norm of $P_{0}f,P_{1}f,P_{2}f$ are respectively $5,1,0$: each added detail removes exactly its own squared coefficient from the error.

The Haar projection can be expressed using level-$J$ Haar scaling functions, or through a coarse scaling family and the Haar wavelets at intermediate levels. On $[0,1]$ with lowest level zero, that coarse family is the single constant function $\varphi$. On the whole real line, use the integer translates of a coarse Haar scaling function instead; one global constant is not square-integrable. Refinement improves the representation while retaining the earlier coefficients.

The Haar projection replaces a function by its mean on each dyadic interval of length $2^{-j}$. For a function satisfying Lipschitz continuity with constant $L$, every value in a cell differs from its cell mean by at most $L{2}^{-j}$. This bound holds for all points with a consistent endpoint convention, and requires no global L2 norm assumption when the projection is defined by local averages.

In the Gaussian white noise model, estimate a Haar scaling function coefficient by $\int {\varphi }_{J,k}dY$. The resulting Haar projection estimator is an unbiased estimator with independent coefficient errors of variance $1/n$. On each cell it is the observed path increment divided by the cell length. This gives a finite-dimensional estimator without imposing smoothness on the drift.

A multiresolution analysis organizes a function into successively finer approximation spaces. In the usual orthonormal construction on $L^2(\mathbb{R})$, these are closed linear subspaces $(V_j{)}_{j\in \mathbb{Z}}$ withRescaling relates the levels: $f(x)\in V_j$ if and only if $f(2x)\in V_{j+1}$. Integer translates of a scaling function form an orthonormal basis of $V_0$. The detail space is the orthogonal complement of $V_j$ inside $V_{j+1}$, giving $V_{j+1}=V_j\oplus W_j$. Coarse information and new details therefore fit together without overlap in squared L2 norm.

For the Haar wavelet, $V_j$ consists of square-integrable functions constant on cells of length $2^{-j}$. Splitting every cell into two halves creates one new contrast per old cell, the Haar refinement identity. On the bounded example $[0,1]$, the first three spaces have dimensions $1,2,4$: a whole-interval mean, two half-interval means, and four quarter-interval means. The Haar projection diagram displays those three resolutions. This bounded construction starts at level zero; the displayed intersection condition refers to the full family on the real line.

The usefulness is a controlled way to increase resolution. Earlier coarse information remains available while new wavelet coefficients identify what the finer description adds. The wavelet approximation complexity connects this extra resolution to both approximation error and the number of coefficients being estimated. ^[ref]

Put $L=2\pi$ and represent the circle by $[0,L)$. The full orthonormal basis of Haar wavelets includes the constant $H_{\ast }=L^{-1/2}$ and, for every dyadic interval $I$ of length $\ell =L{2}^{-j}$, the normalized detailTake half-open intervals and the corresponding values at their boundaries, so every point belongs to exactly one interval at each level. The constant basis function is necessary: if it were omitted, even $f\equiv 1$ could not satisfy the asserted convergence. We use the full conventional Haar wavelet system in the threshold sum.

Let $A_{J}f$ be the function equal to the mean of $f$ on each level-$J$ dyadic interval. Comparing the means on a parent and its two children proves the Haar refinement identity directly: the difference between the child mean and the parent mean is the appropriate value of $c_I{H}_I$. Inductively this gives the Haar projectionWriting ${\omega }_f(s)={\sup }_{\mathrm{dist}(x,y)\le s}|f(x)-f(y)|$ for the modulus of continuity, the cell-average formula yieldsFor a detail on a cell $I$, its integral is zero. Subtracting $f(x_I)$ for any $x_I\in I$ therefore gives the coefficient estimateIn particular $|c_I|\le 2\Vert f{\Vert }_{\infty }\sqrt{\ell }$, so for every $\delta >0$ only finitely many details can have $|c_I|\ge \delta$. The hard-threshold sumis thus a well-defined finite sum. Its chosen details need not form a complete set of levels; this is why convergence of the Haar projections alone does not prove the result.

Fix $\eta >0$. Choose $J\ge 1$ so that ${\omega }_f(L{2}^{-J})\le \eta$. All finer details satisfy $|c_I|\le \eta \sqrt{L}{2}^{-j/2}$ at level $j\ge J$. For sufficiently small $\delta$, every nonzero coefficient at levels below $J$, and the constant coefficient if nonzero, is retained. Also choose the unique integer $K\ge J$ such thatFor $j>K$ we have $|c_I|\le \eta \sqrt{L}{2}^{-j/2}<\delta$, so no such detail is retained. Consequently $T_{\delta }f$ differs from $A_{K+1}f$ only by omitted details at levels $J$ through $K$. At any point there is at most one supported detail per level, and an omitted detail has magnitude at most $\delta /\sqrt{L{2}^{-j}}$. The geometric bound for omitted Haar details givesSince $\Vert A_{K+1}f-f{\Vert }_{\infty }\le \eta$, we concludeThe right side can be made arbitrarily small, proving uniform hard-threshold convergence of normalized Haar expansions:The square-root scaling imposed by the specified $L^2$ normalization is what makes the omitted contributions a controlled geometric series.

Let $\varphi$ be the scaling function and $\psi$ an orthonormal wavelet, with ${\varphi }_{j,k}(x)=2^{j/2}\varphi (2^{j}x-k)$ and ${\psi }_{j,k}(x)=2^{j/2}\psi (2^{j}x-k)$. For $f\in L^2(\mathbb{R})$, the wavelet series at a coarse level $j_0$ isThe series converges in the L2 norm. Its coarse part captures broad variation and successive wavelet levels add finer detail. Truncating at levels $j<J$ produces the orthogonal projection onto $V_J$; Parseval identity makes its squared approximation error the sum of the omitted squared coefficients. Merely square-integrable functions need not have pointwise convergence.

For the Haar wavelet, use the cell convention $I_{j,k}=(k{2}^{-j},(k+1)2^{-j}]$ and let $I_j(x)$ be the unique such interval containing $x$. The Haar scaling functions are $2^{j/2}{1}_{I_{j,k}}$, so the Haar projection isFor a merely locally integrable function this cell-average formula remains defined, even if a global $L^2$ orthogonal projection is unavailable. If $\Vert f^{\prime }{\Vert }_{\infty }=L$, the mean value theorem gives Lipschitz continuity and, since $|t-x|\le 2^{-j}$ on the cell,Thus the requested bound holds with $\boxed{c=\Vert f^{\prime }{\Vert }_{\infty }}$, by the Haar projection error for a Lipschitz function.

To estimate a density, replace each scaling or wavelet coefficient $\int f{\varphi }_{j,k}$ or $\int f{\psi }_{j,k}$ by the sample average of that basis function. The finite-resolution Haar density estimator is thereforeIt is the histogram on the dyadic cells, is nonnegative, and integrates to one. Its expectation is $K_{j}f(x)$. A density with bounded derivative is bounded: the Lipschitz density height bound gives $f(x{)}^2\le L$ by integrating the triangular lower envelope $(f(x)-L|t-x|{)}_{+}$. The case $L=0$ cannot be a probability density on the whole real line. Hence $B=\Vert f{\Vert }_{\infty }<\infty$.

The cell probability is $p_j(x)={\int }_{I_j(x)}f\le B{2}^{-j}$. The sample count has a binomial distribution, soThe Cauchy-Schwarz inequality bounds the mean centered absolute deviation by the square root of this variance. Adding the projection bias givesChoose $j_n=\lfloor ({\log }_{2}n)/3\rfloor$, so $2^{j_n}\asymp n^{1/3}$. The two terms have the same order, andThis is a bias-variance tradeoff: refining the cells decreases the averaging bias but increases the sampling variance.

The Haar projection averages this normal distribution density on each cell; the finer cells follow its variation more closely. A Haar density estimator replaces these exact cell probabilities by empirical frequencies and therefore also introduces sampling noise.

In the Gaussian white noise model, observe the entire pathwhere $W$ is standard Brownian motion and the deterministic drift belongs to L2 space on $[0,1]$. Equivalently, $dY(t)=f(t)dt+n^{-1/2}dW(t)$. For every deterministic $h\in L^2[0,1]$, the observed stochastic integral satisfiesThe noise $W(h)=\int hdW$ is an isonormal Gaussian process. In particular its variance is $\Vert h{\Vert }_2^2$. Gaussian white noise is interpreted through these integrals, rather than as an ordinary random function with a pointwise value at every time. No smoothness of $f$ is needed.

Here is the Gaussian maximum bound without independence. Put $M={\max }_{1\le i\le N}|g_i|$. It is integrable since it is bounded by ${\sum }_i|g_i|$. For every $\lambda >0$,The second inequality uses only the moment-generating function of the standard normal distribution, so independence is unnecessary. Jensen inequality gives $e^{\lambda EM}\le E{e}^{\lambda M}$, henceMinimizing at $\lambda =\sqrt{2\log (2N)}$ proves

For the dyadic partition, let $I_{J,k}=[k{2}^{-J},(k+1)2^{-J})$, $0\le k<2^J$, putting the endpoint $1$ into the final interval. The Haar scaling functionsform an orthonormal basis of the space $V_J$ of functions constant on each interval. The Haar wavelets can be written asFor $J\ge 1$, the constant function ${\varphi }_{0,0}=1$ together with ${\psi }_{j,k}$, $0\le j<J$, is another orthonormal basis of $V_J$. Indeed, each successive level splits a cell's two constants into their sum and difference; the number of basis elements is $1+{\sum }_{j=0}^{J-1}{2}^j=2^J$. Thus the Haar projection is the orthogonal projectionOn cell $I_{J,k}$ its value is $2^J{\int }_{I_{J,k}}f(t)dt$. The first formula also applies to $J=0$.

Estimate each coefficient by its observed stochastic integral:This is the Haar projection estimator in Gaussian white noise. The noise integrals over disjoint intervals form a Gaussian vector with zero off-diagonal covariance. The principle that uncorrelated jointly Gaussian variables are independent then makes these coefficients independent. ThereforeTaking expectations shows that ${\hat{\Pi }}_{V_J}f$ is an unbiased estimator of ${\Pi }_{V_J}f$, both coefficientwise and pointwise for the stated step-function representatives.

Only one scaling function is nonzero in each cell, so the supremum norm of the error has the exact formThe endpoint convention ensures the same identity at $1$. Apply the Gaussian maximum bound without independence with $N=2^J$ to obtain the supremum norm risk of a Haar projection estimator:The factor $2^{J/2}$ comes from cellwise scaling, while the extra square root of $J+1$ comes from taking the maximum over $2^J$ Gaussian errors.