Noise-limited detection threshold ($T=k\sigma$)

A noise-limited detection threshold is the decision boundary above which a measured statistic is declared to contain a signal, with the boundary chosen from a specified noise model and decision criterion. It belongs to a measurement procedure: filtering, observation time, baseline correction and the definition of a positive detection affect the result. A threshold in observed data is distinct from the true signal level required for a specified detection probability.

For an original one-sided Gaussian model, let $Y=A+N$, where $A\ge 0$ is the signal and $N\sim \mathcal{N}(0,{\sigma }^2)$. Declare a detection when $Y>T$ and choose $T=k\sigma$. If $\Phi$ is the cumulative distribution function of the standard normal distribution, thenHere $\alpha$ is the per-observation false alarm rate. The model assumes the same additive noise distribution with and without the signal.

With $k=5$, the noise-only crossing probability is approximately $2.87\times 10^{-7}$ for one independent observation. Yet a signal of size $A=5\sigma$ is detected only half the time: it puts the distribution's center exactly at the decision boundary. “Five times noise” therefore does not by itself mean reliable detection of a signal that large.

To require both a chosen false-positive probability $\alpha$ and a missed-detection probability $\beta$, this model givesFor $\alpha =\beta =0.05$, the two values are approximately $1.645\sigma$ and $3.290\sigma$. This distinction accords with the VIM definition of a limit of detection, which specifies both kinds of decision error rather than equating the limit with the observed threshold.^[ref]

The experiment should measure baseline fluctuations under the same bandwidth, loading and observation rule used for signal detection, then test known signal levels. Repeated searching also matters: for $M$ independent noise-only observations, the chance of at least one threshold crossing is $1-(1-\alpha {)}^M$. Correlated samples require a different calculation. Baseline drift, non-Gaussian fluctuations and signal-dependent noise likewise invalidate a threshold inferred from the simple model without further characterization.