= Scalloping loss
<Scalloping loss> is the underestimate of an isolated sinusoidal amplitude caused by measuring it only at the nearest sampled <discrete Fourier transform> <frequency>, after correcting the window’s coherent amplitude scale. An off-grid tone is evaluated away from the maximum of the window response. This differs from the <coherent gain of a window>, which describes the response at the correctly centered <frequency>. The instrument manual treats amplitude scaling and off-bin errors separately.<LabVIEW Analysis Concepts by National Instruments>[printed p. 5-23, Scaling Smoothing Windows and Table 5-3, off-center amplitude errors]{ref}
Let $w_n$ be a finite real window with positive sum, and let $x_n=Ae^{2\pi i(k+\delta)n/N}$, where $k$ is the nearest unpadded DFT bin and $|\delta|\le1/2$. Write $W(\nu)=\sum_{n=0}^{N-1}w_ne^{-2\pi in\nu}$. The coherent-gain-corrected bin magnitude relative to $|A|$ is
$$
r(\delta)=\frac{|W(\delta/N)|}{W(0)},\qquad
L(\delta)=-20\log_{10}r(\delta)\ \mathrm{dB}.
$$
Here $L$ is a positive loss when $r<1$. The <half-bin amplitude response of a window> evaluates this ratio at the largest nearest-bin offset. Other tones, noise and the negative-<frequency> image of a real sinusoid can also affect a measured peak; the derivation isolates one complex tone.
For a <rectangular window>, its geometric-sum transform gives
$$
r(\delta)=\frac{|\sin(\pi\delta)|}{N|\sin(\pi\delta/N)|},
\qquad r(1/2)=\frac1{N\sin(\pi/(2N))}.
$$
For $N=64$, the half-bin ratio is about $0.63668$ and the loss about $3.92\,\mathrm{dB}$. An amplitude-one tone exactly centered on a bin is recovered as one, while the same tone shifted by half a bin gives only about $0.637$ at either nearest bin. Dividing by coherent gain cannot remove this offset dependence.
For the periodic <Hann window>, use
$$
W_H(\nu)=\tfrac12W_N(\nu)-\tfrac14W_N(\nu-1/N)-\tfrac14W_N(\nu+1/N),
$$
where $W_N$ is the rectangular response. At $N=64$, this gives a half-bin ratio about $0.84883$, or $1.42\,\mathrm{dB}$ loss. The smaller nearest-bin error comes with a wider main lobe and a larger <equivalent noise bandwidth>; window choice affects several measurement properties at once.
<Zero padding> reduces the nearest-grid offset without changing the measured record. Padding to $M=pN$ points makes the nearest offset at most $1/(2p)$ in original-bin units. With $p=4$ and $N=64$, the rectangular response at that worst offset is about $0.97450$, a loss of only $0.224\,\mathrm{dB}$. This is denser sampling of the same <discrete-time Fourier transform>, not extra physical <frequency> resolution. A fitted peak can reduce a grid error too, but its signal model and noise sensitivity must be stated.
A proposed analyzer test holds tone amplitude fixed while sweeping <frequency> through one bin interval, retaining record length, sampling rate and window. Divide out coherent gain, plot the measured peak versus fractional-bin offset, and compare with the exact response above. Include phase and nearby-tone checks for a real signal. Summing spectral powers instead answers an energy question and requires its own bandwidth normalization; it should not silently replace this single-bin amplitude definition.
Back to article page
