O4 (source code)
= O4
{c}
{scope}
Minimizing $\|Y-X\theta\|^2$ gives the normal equations $X^TX\hat\theta=X^TY$. Since $X$ has <matrix rank>[full column rank], $X^TX$ is positive definite and invertible, and the objective has a unique <local minimum>[minimum]. Set $A=(X^TX)^{-1}X^T$; then $\hat\theta=AY$ and $AX=I_p$. The model gives $\mathbb E[Y]=X\theta$ and $\operatorname{Var}(Y)=\sigma^2I_n$, so $\mathbb E[\hat\theta]=\theta$ and $\operatorname{Var}(\hat\theta)=\sigma^2AA^T=\sigma^2(X^TX)^{-1}$. For the alternative estimator the corresponding general formulas are $\mathbb E[\tilde\theta]=DX\theta$ and $\operatorname{Var}(\tilde\theta)=\sigma^2DD^T$.
Back to article page
