Question

Consider the following scenario:

Researchers place unobtrusive video recording devices in the large lecture classrooms around campus. Later, coders view tapes of the classrooms and record how often students in each classroom checked their phones during class. The researchers are interested in seeing if students who check their phones more often get worse grades in class. Later on, they connected the “phone” score to the students’ grades to look for a correlation.

1) What type of measure is being used for the “checking phones” variable (self-report, observational, or physiological)?

2) What scale of measurement (categorical, ordinal, interval, or ratio) is being used for the “checking phones” variable?

3) What kind(s) of reliability would need to be evaluated for the phone measure?

4) Draw a scatterplot showing that the “checking phones” measure has good test-retest reliability. (You do not have to turn in the drawing). (a) What should the labels for the x-axis and y-axis be if the plot is showing test-retest reliability? (b) What type of relationship would we expect to be shown in our scatterplot if the “checking phones” measure has good test-retest reliability (e.g., strong vs. weak, positive vs. negative relationship)? (c) In your scatterplot, what does one dot represent?

Answer 1

1) The observational measure is being used in this case, as video clips are being used to obtain data.

2) The "checking phone" is a ratio scale of measurement, as it can have a true zero point.

3) The test-retest reliability would be required for the phone measure so that it can be evaluated whether students are displaying similar behavior in other classes as well.

4) (a) The x-axis would be Phone checking (T1) and the y-axis would be Phone Checking (T2), with T1 representing the results of the first instance when the measure was taken and T2 indicating the second time.

(b) For a good test-retest reliability, we could expect the scatter plot to show a 45 degrees inclination, with the dots scattered more or less around the straight line. This would be interpreted as a positive and strong relationship.

(c) One dot in the scatter plot would indicate one participant.