Statistical significance

A result is statistically significant at a specified significance level $\alpha$ when its valid p-value is at most $\alpha$ and the corresponding null hypothesis is rejected. Significance does not measure effect size, establish practical importance, or give the probability that the null is true. Software stars encode thresholds for $p$-values; they do not supply an additional statistic.

For a known background rate $b$ observed for exposure $T$, independent events give a Poisson distribution with mean ${\mu }_b=bT$. An excess-count test of the background-only null hypothesis uses the upper-tail P-value in the title. The tail, rather than the probability of exactly $n$, counts all outcomes at least as discrepant in the chosen direction.

This probability is conditional on the null model; it is not the posterior probability that the null model is true. A stipulated discovery threshold is met only when the tail is below that threshold. Uncertainty in the background rate or selecting among many searches needs additional modelling rather than being absorbed into the same known-mean formula.

Searching many possible signal locations increases the chance of finding an apparently unusual fluctuation somewhere. A local p-value for a specified location differs from the global p-value for the search over locations.^[ref]