Fixed-design nonparametric regression

The fixed-design nonparametric regression model observes $Y_i=m(x_i)+\sigma {ϵ}_i$ at deterministic design points $x_i$. For independent and identically distributed random variables ${ϵ}_i$ with the standard normal distribution, comparing two mean functions has Kullback-Leibler divergence ${\sum }_i(m_1(x_i)-m_0(x_i){)}^2/(2{\sigma }^2)$.

At design sites $i/n$, let ${\tilde{Y}}_n$ be the step function taking response $Y_i$ on $((i-1)/n,i/n]$. Project it onto an interval-adapted wavelet basis through resolution $J$. Each coefficient is ${\hat{c}}_{\lambda }={\sum }_i{Y}_i{\int }_{I_i}{b}_{\lambda }(t)dt$. Its expectation is the corresponding coefficient of the step function of sampled regression means. For a Haar wavelet basis, dyadic sample size and resolution at most the sampling scale, this is exactly $n^{-1}{\sum }_i{Y}_i{b}_{\lambda }(i/n)$ with right-closed pointwise versions. Independent errors of variance ${\sigma }^2$ give coefficient variance at most ${\sigma }^2/n$: Cauchy-Schwarz inequality on each sampling cell yields ${\sum }_i({\int }_{I_i}{b}_{\lambda }{)}^2\le \Vert b_{\lambda }{\Vert }_2^2/n$.

Hard thresholding keeps detail coefficients whose magnitude exceeds a prescribed threshold and sets the rest to zero, while retaining the coarse approximation coefficients. Soft thresholding instead uses $\mathrm{sgn}(\hat{b})(|\hat{b}|-\tau {)}_{+}$. The simultaneous wavelet coefficient noise bound motivates thresholds of order $\sigma \sqrt{\log n/n}$ in a normalized Gaussian regression model with at most order $n$ details. This is a nonlinear selection rule: coefficients can be removed individually rather than discarding an entire finest resolution.

For coefficient weights $w_{\lambda i}$ and independent mean-zero sub-Gaussian random variables of parameter $\sigma$, put $q_{\lambda }^2={\sum }_i{w}_{\lambda i}^2$. Multiplying moment-generating-function bounds shows that the weighted noise has parameter $\sigma q_{\lambda }$. The exponential Markov inequality, optimized in its parameter, gives tail bound $2\exp (-t^2/(2{\sigma }^2{q}_{\lambda }^2))$. At the displayed thresholds, a union bound over $N$ coefficients gives simultaneous control with probability at least $1-\delta$. No coefficient independence is required. Integrated wavelet weights satisfy $q_{\lambda }\le n^{-1/2}$. Finite variance alone only supports the weaker Chebyshev choice ${\tau }_{\lambda }=\sigma q_{\lambda }\sqrt{N/\delta }$.

Let $x_i=i/n$, $1\le i\le n$, and let $N_x$ count sites within distance $h/2$ of $x\in [0,1]$. If $nh\ge 2$, a nearest site lies in the closed window, so $N_x\ge 1$. A clipped window has length at least $\min (h,1)/2$, and a grid count gives $N_x\ge n\min (h,1)/2-1$. Combining these two bounds yields $N_x\ge n\min (h,1)/4$. For $0<h\le 2$, the stronger clipped-length bound $h/2$ directly gives $N_x\ge \lfloor nh/2\rfloor \ge nh/4$. These estimates include both endpoints of the domain.

At an interior target point in regular fixed-design nonparametric regression, mean functions with Lipschitz bound $L$ have pointwise minimax risk of order $(L{\sigma }^2/n{)}^{2/3}$ for fixed positive $L,\sigma$ as $n\to \infty$. A smoothing bandwidth of order $({\sigma }^2/(n{L}^2){)}^{1/3}$ balances squared bias of an estimator and variance; the two-point lower bound with a triangular bump gives the matching lower rate.