Question

A researcher gives males and female students a 20 item open-ended response test about knowledge of statistics where every item is worth 10 points. The researcher then conducts 20 independent sample t-tests and, using an alpha of .05, finds a significant difference for 2 items. (a) What problem does this researcher face? (b) How could they address the problem?

Answer 1

Solution:

a)

• At level of notable of 0.05 ,the probability of type-1 error is 0.05;This method we anticipate to find the wrong against notable difference 5% of time.
• since these were 20 tests, we anticipate to find 0.05*20=1 of them notable even when there is no real difference.

b)

• Firstly only those tests must be manage which make a reasoning sense before drawing the sample.
• Simply testing all feasible differences in order to find a statistically significant one is called P-hacking, and is an unfair rehearse.
• Secondly, a bottom significance level must be used, so that probability of acquire at least one significant test out of 20 is very small.