Past exam of the physics A course of the University of Cambridge / 2010 / Paper 2 / Section D / D12 / a / Solution

For independent Gaussian measurements with correctly specified standard deviations, model residuals give the chi-squared test statisticFor a linear fit of $p$ identifiable parameters under the null model, it has a chi-squared distribution with $\nu =n-p$ degrees of freedom. The upper-tail p-value $P({\chi }_{\nu }^2\ge X_{\mathrm{obs}}^2)$ tests unusually large residuals; a nonlinear fit generally uses this distribution only approximately. Correlated Gaussian errors instead require $X^2={\mathbf{r}}^{\mathsf{T}}{\Sigma }^{-1}\mathbf{r}$.

For counts, Pearson's chi-squared test uses $X^2={\sum }_j(O_j-E_j{)}^2/E_j$, with $E_j$ calculated under the null hypothesis. For $k$ multinomial categories and $p$ fitted parameters the usual asymptotic degrees of freedom are $k-1-p$; the subtraction of one reflects the fixed total count. Sparse expected counts can invalidate this approximation, requiring a suitable exact or simulation-based test. Contingency-table independence gives $(r-1)(c-1)$ degrees of freedom.

A small p-value indicates tension with the specified model and error assumptions; it is not the probability that the null is true. Excessive $X^2$ may reflect a wrong model, underestimated errors, correlations or outliers. An implausibly small value may suggest overestimated errors or hidden constraints. For an adequate fitted model $X^2/\nu$ is typically near one, with fluctuations, rather than required to equal one.