Solution

ID: past-exam-of-the-physics-a-course-of-the-university-of-cambridge/2019/paper-1/section-d/d13/a/solution

A chi-squared test compares squared deviations with the size expected from the assumed random errors. For independent observations with a normal distribution with known standard deviations and predictions , use
For a correct fixed model, the standardized independent residuals are unit-variance normal distributions, so has a chi-squared distribution with . Fitting independent linear parameters removes residual directions, giving . This distribution is exact for a full-rank linear model with known errors with a normal distribution; it is an approximation for many nonlinear fits. With known correlated errors with a multivariate normal distribution, replace the sum by , where is the covariance matrix and its rank and fitted constraints determine the available statistical degrees of freedom.
Under the null hypothesis, and . The reduced statistic should therefore be near one, with standard deviation , rather than equal to one exactly. An upper-tail p-value is
Compare it with a significance threshold chosen in advance. A small value indicates tension with the model together with its error assumptions; it can arise from a wrong mean model, underestimated uncertainties or neglected correlations. An unusually small statistic can suggest overestimated uncertainties, correlations or overfitting. A p-value is not the probability that the model is true, and failure to reject is not proof of the model.
For counts in categories, the Pearson form is . Its chi-squared distribution is an asymptotic approximation requiring adequate expected counts. For a fixed-total multinomial sample there is one normalization constraint, so the usual statistical degrees of freedom are after fitting other parameters. Independent Poisson bins instead start with statistical degrees of freedom before parameters are fitted. Sparse counts require an appropriate count likelihood or a calibrated exact or simulation-based test; plugging tiny expected counts into the asymptotic formula is unreliable. The error model, binning, fitted parameters and chosen test must therefore be reported with the result.

New to topics? Read the docs here!