Question

Researchers were interested in how belief in psychic abilities affected performance on a card matching activity. Participants were divided into four groups based on their reported level of belief:

Group 1: No belief
Group 2: Some belief, but skeptical
Group 3: Some belief, but open-minded
Group 4: Full belief

Then for each participant, a researcher drew 20 cards from the top of a standard deck of cards, and asked the participant to guess the suit of each card. For each participant, the number of successes was recorded.

The researchers performed ANOVA to determine if there were any differences in the mean number of successes among the groups. Their F statistic was F = 4.34.

A second group of researchers did an exact replication of the first study - the procedure, and sample sizes in each group, were exactly the same as the first study. The second study had an F statistic of F = 2.12.

First study: F = 4.34
Second study: F = 2.12

Which of the following statements about the two studies are true?

- Both studies had the same value of MSG/MSE (MSG divided by MSE).

- It is impossible to compare the p-values in these studies.

- The second study found stronger evidence against the null hypothesis than the first study found.

- The second study found weaker evidence against the null hypothesis than the first study found.

- In the second study, MSG/MSE (MSG divided by MSE) was larger than in the first study.

- The p-value in the second study is smaller than the p-value in the first study.

- In the second study, MSG/MSE (MSG divided by MSE) was smaller than in the first study.

- The p-value in the second study is larger than the p-value in the first study.

Answer 1

This questions can be answered using the assumptions, formulaes and distribution theory used in anova.

1. False - F statistic = MSG/MSE. F is different in the two cases.

2. False - p values can be calculated as anova assumption of observations follow normal distribution.

3. False. Larger the value of F, smaller it supports the null hypothesis.

4. True - larger the value of F, strongly it opposes the null hypothesis.

5. False - F value in second case is smaller than that in first case

6. False - p-value works opposite to the statistic value. ie. smaller the value of p-value, strongly it opposes to the null hypothesis. p-value will be larger in the second case.

7. True - F value is smaller in second case.

8. True - reason is same as in 6.