Question

For multiple linear regression, two cases of models exist in this problem.

Y -> overall test score / ex) sat

A,B,C,D,E -> subjects of the test

First Model:
Y = A + B + C + D   / intercept <0, coefficients: β(A)<0,  β(B)<0,  β(C)>0,  β(D)>0

Second Model:
Y = B + C + D (excluded A) / intercept <0, coefficients:  β(B)>0,  β(C)>0,  β(D)>0

At the first model, estimated coefficient of A and B were negative. The result was quite confusing, so second model was made from eliminating A variable.

(It is expected to have positive coefficients of specific subjects' score for overall test score)

Question is, what was the possible causes of potential problem with First Model.

What would be the problem of this confusing result of negative coefficients of A??

Answer 1

A negative coefficient in the regression model indicates that as the value of the variable increases the value of the dependent variable decreases thereby having a negative impact.

Model 1 : coefficients: β(A)<0,  β(B)<0,  β(C)>0,  β(D)>0

Model 2 : coefficients:  β(B)>0,  β(C)>0,  β(D)>0 ( Eliminating variable A)

Thus, by eliminating the variable A, all the variables have a positive coefficient. The possible reason for this change is listed below :

1. Variable A had outliers (not realistic values)

2. Variables A & B would have been correlated