Past exam of the mathematics course of the University of Cambridge / 2007 / ib / Paper 4 / 19C / iii

Set $\mathbf{e}=1/\sqrt{n}$ and $\mathbf{f}=\mathbf{x}/\sqrt{S_{xx}}$. These are orthonormal because ${\sum }_i{x}_i=0$. Complete them to an orthonormal basis $\mathbf{e},\mathbf{f},{\mathbf{q}}_3,\dots ,{\mathbf{q}}_n$ of ${\mathbb{R}}^n$. An orthogonal transformation of the standard normal error vector $\mathbf{\epsilon }/\sigma$ has independent normal random variables as coordinates $Z_1,\dots ,Z_n$: its joint probability density remains $(2\pi {)}^{-n/2}\exp (-\Vert z{\Vert }^2/2)$ because the transformation preserves squared length and has Jacobian determinant of absolute value one.

Projection onto these basis directions givesTherefore the joint distribution of least-squares and variance estimators isThe last law is the chi-squared distribution of a sum of $n-2$ squared independent normal random variables with mean zero and variance one, and independence follows because the three statistics use disjoint independent coordinate sets. When all $x_i=0$, omit the slope direction: $\hat{\alpha }$ and $n{\hat{\sigma }}^2/{\sigma }^2\sim {\chi }_{n-1}^2$ are independent, and there is no identifiable slope estimator.