Multicollinearity means that predictor columns are strongly linearly related. It inflates coefficient variances and can make individual effects imprecise even when a joint test strongly rejects that all corresponding coefficients vanish.
For a slope in a regression with an intercept, the variance inflation factor is , where comes from regressing its predictor on all the others. With one other predictor it is , quantifying the increase in conditional estimation variance.
For two centered predictors with squared norms and correlation coefficient , the ordinary least squares estimate of either partial slope has variance . Compared with the same predictor alone and the same error variance, the inflation factor is . The eigenvalues of the slope Gram matrix are and .
Articles by others on the same topic
Multicollinearity refers to a situation in multiple regression analysis where two or more independent variables are highly correlated with each other. This high correlation can lead to difficulties in estimating the coefficients of the regression model accurately. When multicollinearity is present, the following issues can occur: 1. **Inflated Standard Errors**: The presence of multicollinearity increases the standard errors of the coefficient estimates, which can make it harder to determine the significance of individual predictors.