Question

A published paper compares the lengths of three different bird species; A, B and C. An ANOVA rejects the null hypothesis (A=B=C) at p=0.0001. Your colleague shows that paper as proof that A and B are significantly different. Give at least one reason that he may be wrong.

Answer 1

The hypothesis for carrying out ANOVA is defined as:

Choose a level of significance, say .

If p-value < , then the null hypothesis is rejected, otherwise not.

In the given problem, there are 3 treatments, namely A, B and C. It is given that the null hypothesis has been rejected as the p-value = 0.0001 is too low.

This implies that the alternate hypothesis is accepted, that is atleast one of the treatment means is different from the remaining.

There could be a possibility that the difference between B and C is insignificant, however, the difference between A and B is significant.

It could also be possible that the all the three treatment means are different.

Thus, the colleagues interpretation may be wrong as the null hypothesis would be rejected even in case where the difference between the treatment means A and B may not be significant, however, they might be significantly different from C. There is possibility that the colleagues statement is correct, but, this can not be proved until Post-Hoc analysis have been carried out.