Question

An important part of using statistics is being able to explain your results to decision makers. Imagine that you have conducted a two-sample test and determined that the difference was not statistically significant. While one mean was 4.3 and the other was 3.9, the p level for the test was p=.07. Your management team says "well the difference may not be statistically significant, but the difference is there! Discuss how you would respond and how you would explain the purpose of the test and significance in this case.   

Answer 1

There is a clear difference of 4.3 - 3.9 = 0.4 in the 2 means, but as the p-value for the test of significance for the difference between 2 means is 0.07 > 0.05, therefore the test is not significant which means that the difference in the 2 means is only by chance and not really there. For the test, the null and alternate hypothesis are given as:

The p-value is defined as the probability of getting the required result given that H₀ is true. Now here as the p-value is large enough, therefore we conclude that the null hypothesis cannot be rejected and therefore the difference is not significant in this case.