Question

1. A researcher notes a reliability score of 0.75 between two observers in a clinical drug trial evaluating whether the drug decreases anxiety. This score means that:
   1. there is 25% random error.
   2. there is 75% random error.
   3. this is an acceptable measure of reliability
   4. this measurement tool is not reliable.

Answer 1

A. There is 25 % random error

Reliability score is .75

Reliability is the degree to which the measure of a construct is consistent or dependable.

Reliability score 0.75 measure indicates 75% reliability and 25% random error.

Generally, interrater reliability is best at 0.90, and any value less than 0.80 should generate serious concern about the reliability of the data or the data gatherer.

According to Cohen's original article, values ≤ 0 as indicating no agreement and 0.01–0.20 as none to slight, 0.21–0.40 as fair, 0.41– 0.60 as moderate, 0.61–0.80 as substantial, and 0.81–1.00 as almost perfect agreement. McHugh says that many texts recommend 80% agreement as the minimum acceptable interrater agreement.