Question

If you run summary() command on a result of linear model fitting returned by lm(), you will see a column Pr(>t). You could guess that it is a result of some t-test. How does it apply here? So far we have seen t-tests in the settings of testing sample locations. The ingredients were: 1) null hypothesis: in earlier cases we looked into the null that stated that two samples came from the distribution(s) with the same mean(s); 2) test statistic: for most examples so far we chose to look at the difference between the means of the two samples - maybe normalized by the (sample) standard deviation, if we are talking about t-test; 3) some calculation (either brute force resampling or analytical t-distribution) that told us whether the observed difference between the means is likely to be observed merely by random chance. That latter probability for the test statistic to reach the observed, or even larger, value under the null hypothesis is the p-value, which directly described the significance of the finding. In connection with the linear model y=ax+b+e (e is the random term), our findings that we want to characterize statistically are coefficients a,b (we just compute them straightforwardly from the data). Describe qualitatively: (1) why do we need statistical description (significance and all) in the first place? We got data, they do not lie exactly on a straight line, but we calculated least square line and that's the best fit we can get. Where is the randomness against which we want to calculate significance of our coefficients? (2) significance involves the null hypothesis. What do you think would be the null hypothesis in this case? It may help you to think what finding would be insignificant - what is your common sense telling you? (3) Now that we understand the source of randomness and formulated the null hypothesis, how would we test it? Again, I am looking for qualitative understanding only; if you think you see how an analytical test (e.g. t-test) could be applied, that's great; if you can describe (very briefly, a few sentences) the brute force sampling procedure that you would use to calculate the significance of the regression coefficient, e.g. slope, it's fine too. Again, I am not looking for math, but for the discussion of why linear regression is just another statistical problem, to which all the usual statistical ways of thinking should apply (randomness, underlying distributions vs estimators, significance levels, hypothesis testing).

Answer 1