Past exam of the physics A course of the University of Cambridge / 2012 / Paper 1 / Section D / D12 / a

A chi-squared test compares discrepancies with their expected random size. For independent Gaussian observations with known standard deviations, a fitted model givesUnder a correct regular model with $p$ fitted parameters, its reference statistical degrees of freedom are usually $\nu =n-p$. Correlated errors instead require ${\chi }^2={\mathbf{r}}^T{C}^{-1}\mathbf{r}$ with the correct covariance matrix. The reduced statistic has mean about one and standard deviation $\sqrt{2/\nu }$, so it should not be forced to equal exactly one.

For binned counts, Pearson's statistic is ${\sum }_i(O_i-E_i{)}^2/E_i$. With a fixed total and $k$ populated categories, fitting $p$ distribution parameters generally gives $\nu =k-1-p$. The approximation needs adequately populated expected bins and the appropriate dependence assumptions. Sparse data may require a Poisson likelihood, exact calculation or simulated reference distribution rather than a casual chi-squared approximation.

The upper-tail probability $p_{\mathrm{value}}=P({\chi }_{\nu }^2\ge {\chi }_{\mathrm{obs}}^2)$ measures how unusual the discrepancy would be under the null model; it is not the probability that the model is true. A large statistic can indicate a wrong model, underestimated noise or unmodelled systematic effects; an unusually small one can indicate overestimated errors or correlations. Residual plots and checks of the noise model are needed alongside the numerical test.