Half-bin amplitude response of a window (source code)

= Half-bin amplitude response of a window
{title2=$r_{1/2}=|W(1/(2N))|/|W(0)|$}

The <half-bin amplitude response of a window> is the magnitude response to an isolated tone halfway between neighboring <discrete Fourier transform> <frequencies>, divided by the response to a bin-centered tone of the same amplitude. It quantifies a particular off-grid amplitude error after correcting the <coherent gain of a window>. For coefficients $w_0,\ldots,w_{N-1}$ with nonzero sum, define
$$
W(\nu)=\sum_{n=0}^{N-1}w_ne^{-2\pi in\nu},\qquad
\boxed{r_{1/2}=\frac{|W(1/(2N))|}{|W(0)|}.}
$$
A loss convention reports $L_{1/2}=-20\log_{10}r_{1/2}$ in decibels. The primary instrument manual tabulates approximately $3.92\,\mathrm{dB}$ for rectangular weighting and $1.42\,\mathrm{dB}$ for Hanning weighting.<LabVIEW Analysis Concepts by National Instruments>[printed p. 5-23, Table 5-3, Worst-Case Amplitude Error]{ref}

To derive the ratio, take a complex tone $x_n=Ae^{2\pi i(k+1/2)n/N}$. Evaluating the windowed transform at bin $k$ gives $A W(-1/(2N))$. For real window coefficients, the responses at opposite offsets have equal magnitude. Dividing by $|W(0)|$ removes the on-bin scaling and leaves $|A|r_{1/2}$. This calculation isolates the <frequency>-grid effect from noise and other spectral components.

For a <rectangular window>, summing the geometric series yields
$$
r_{1/2}=\frac1{N\sin(\pi/(2N))},\qquad
\lim_{N\to\infty}r_{1/2}=\frac2\pi.
$$
For $N=64$, the ratio is about $0.63668$. The large-$N$ value is about $0.63662$, explaining the familiar $3.92\,\mathrm{dB}$ <scalloping loss> rather than an exact finite-record constant.

For the periodic <Hann window>, $w_n=(1-\cos(2\pi n/N))/2$. Its finite response is
$$
W_H(\nu)=\tfrac12W_N(\nu)-\tfrac14W_N(\nu-1/N)-\tfrac14W_N(\nu+1/N),
$$
where $W_N$ is the rectangular response and $W_H(0)=N/2$ for $N>1$. Substitution gives about $0.84883$ at $N=64$. A useful limiting derivation replaces the normalized sum by its integral:
$$
\lim_{N\to\infty}\frac{W_H(x/N)}{N/2}
=\int_0^1[1-\cos(2\pi t)]e^{-2\pi ixt}\,dt
=e^{-\pi ix}\frac{\operatorname{sinc}(x)}{1-x^2},
$$
with continuous limiting values at $x=\pm1$. At $x=1/2$ its magnitude is $8/(3\pi)\approx0.84883$. For a short $N=4$ periodic window, the coefficients $(0,1/2,1,1/2)$ instead give $r_{1/2}=(1+\cos(\pi/4))/2\approx0.85355$. The finite definition avoids treating the asymptotic value as exact.

The symmetric <Hann window> uses denominator $N-1$, not $N$; it has a different finite response and coefficient sum. SciPy explicitly distinguishes the symmetric filter-design convention from the periodic spectral convention.<SciPy 1.16.2 documentation>[scipy.signal.windows.hann, Parameters: sym; Returns; Notes]{ref} The exact half-bin ratio should therefore use the coefficients actually applied.

For an original acquisition example, a $64$-sample record at $6400\,\mathrm{Hz}$ has $100\,\mathrm{Hz}$ bin spacing. A complex tone at $1050\,\mathrm{Hz}$ lies halfway between bins 10 and 11. An amplitude-one tone gives a corrected nearest-bin magnitude about $0.637$ with rectangular weighting or $0.849$ with periodic Hann weighting. <Zero padding> changes the sampled grid of the same <discrete-time Fourier transform>; if the padded grid contains $1050\,\mathrm{Hz}$, its value there has no half-bin error relative to that tone.

The half-bin response is not automatically a universal worst-case bound for every conceivable window. Nor does the isolated-complex-tone calculation cover every real-sinusoid measurement: near zero <frequency> or the sampling limit, the mirrored spectral component can overlap and make the observed magnitude phase-dependent. A proposed test sweeps a constant-amplitude tone through a bin interval, varies phase for real signals, and compares the complete finite-sum response before quoting a single off-bin accuracy number.