Solution (source code)

= Solution

Let $v_i=[(X^TX)^{-1}]_{ii}>0$. Part (b) gives
$$
Z_i=\frac{\widehat\theta_i-\theta_i}{\sigma\sqrt{v_i}}\sim N(0,1).
$$
The residual vector and coefficient estimator are independent: they are jointly <multivariate normal distribution>[normal] and their cross-<covariance matrix> is zero, since $X^T(I-H)=0$. Hence $Z_i$ is independent of $U=(n-p)s^2/\sigma^2\sim\chi^2_{n-p}$. By the definition of <Student's t-distribution>,
$$
\frac{\widehat\theta_i-\theta_i}{s\sqrt{v_i}}
=\frac{Z_i}{\sqrt{U/(n-p)}}\sim t_{n-p}.
$$
Let $t_{n-p,1-\alpha/2}$ be the $1-\alpha/2$ quantile of this distribution. Inverting the central <probability> statement gives the exact marginal <confidence interval>
$$
\boxed{\widehat\theta_i\ \pm\ t_{n-p,1-\alpha/2}\,s\sqrt{[(X^TX)^{-1}]_{ii}}.}
$$
Its coverage is $1-\alpha$ under the stated <Gaussian linear model>; simultaneous coverage for all coefficients would require a separate adjustment.