 Solution

A chi-squared test compares residuals with the random scatter predicted by a model. For independent measurements $y_i$ with known Gaussian standard deviations ${\sigma }_i$ and predictions ${\mu }_i(\theta )$, formIf the mean model and error assumptions are correct, a full-rank linear fit of $p$ parameters gives a chi-squared distribution with $\nu =N-p$ statistical degrees of freedom. With a fully specified prediction and no fitted parameters, $\nu =N$. Nonlinear fits usually make this an approximation. Independent external calibration is not the same as fitting a parameter to these test data.

Under the reference distribution, the mean statistic is $\nu$ and its variance is $2\nu$. The upper-tail p-value $\mathrm{Pr}({\chi }_{\nu }^2\ge Q_{\mathrm{obs}})$ quantifies how unusual a discrepancy at least this large would be under the assumed model. A small value suggests model failure or incorrect uncertainties; it is not the probability that the model is true. An unusually small statistic can also reveal inflated uncertainties or correlations. A rejection threshold should be specified in advance.

For independent Poisson distribution counts, the model variance is its expected count ${\mu }_i$. The Pearson chi-squared statistic is $\sum (y_i-{\mu }_i{)}^2/{\mu }_i$. The estimated-error alternative ${\sigma }_i\simeq \sqrt{y_i}$ is common for large counts but can differ substantially when counts are small. In that regime the Poisson deviance or a direct likelihood calculation is preferable, with simulation if an accurately calibrated goodness-of-fit probability is required.