Question

why should we remove non-significant variables from multiple linear regression models? What problems may arise if we keep them in the model?

Answer 1

The data are incapable of really telling you which model is "better" unless you use AIC in a highly structured way (e.g. on a pre-specified large group of variables), and removing insignificant variables invalidates the estimate of sigma² and all P-values, standard errors, and confidence limits in addition to invalidating the formula for adjusted R²

The first is prediction accuracy: keeping all variables often have low bias but large variance. Prediction accuracy can sometimes be improved by shrinking or setting some coefficients to zero. By doing so we sacrifice a little bit of bias to reduce the variance of the predicted values, and hence may improve the overall prediction accuracy.

The second reason is interpretation. With a large number of predictors, we often would like to determine a smaller subset that exhibit the strongest effects.