Categories
ajoBlog

On Response Order Effects…

The question was along the lines of “do you think the response order make a difference with Likert-style alternatives?”…

As in:

Strongly Agree, Agree…–>
vs.
Strongly Disagree, Disagree… –>


The order of the scale absolutely makes a difference (Friedman, Herskovitz, & Pollack, 1994), and this is something I found by accident when I taught high school, and would issue culture surveys to the students.

I swapped the scales after a few surveys, and found what many researchers have confirmed – if the scales start off positive, as in “Strongly Agree” the overall results of the survey tended to skew that way, too! The means for those culture surveys rose, to some degree and influence, by the use of alternate scale construction.

I think it has something to do with the way we decode text. Calkins (1970) suggests that reading itself is a series of jumps, and fixations – when your eyes come to rest on the page.

We naturally read, in the western hemisphere, from left to right (Spalek & Hammad, 2005), in a “preferential looking paradigm” (Guo K, Meints K, Hall C, Hall S, & Mills D, 2009), studying the left side of something more intently (page of a book, website, Likert scales).

Turns out, this behavior is not unique to humans – monkeys, dogs, and some birds display the same trait.

When you come across a scale like a Likert-style setup, with the positive answers placed towards the left, or first in order, a human eye simply comes to a rest on the first available, acceptable, alternative.

If the scales were administered to a right-to-left reading respondent (Chokron & Imbert, 1993), I would suspect the results to be the opposite.

 

References

Calkins, E. O. (1970). Reading Forum: A Collection of Reference Papers Concerned with Reading Disability. National Institute of Health.

Chokron, S., & Imbert, M. (1993). Influence of reading habits on line bisection. Cognitive Brain Research, 1(4), 219–222. https://doi.org/10.1016/0926-6410(93)90005-P

Friedman, H. H., Herskovitz, P. J., & Pollack, S. (1994). The biasing effects of scale-checking styles on response to a Likert scale. Proceedings of the American Statistical Association Annual Conference: Survey Research Methods, (1972), 792–795.

Guo K, Meints K, Hall C, Hall S, & Mills D. (2009). Left gaze bias in humans, rhesus monkeys and domestic dogs. Animal Cognition, 12(3), 409–418. Retrieved from http://www.springerlink.com/content/b188227326337268/?p=886c53fb9c8140948185b097451c7f5a&pi=0

Spalek, T. M., & Hammad, S. (2005). The left-to-right bias in inhibition of return is due to the direction of reading. Psychological Science, 16(1), 15–18. https://doi.org/10.1111/j.0956-7976.2005.00774.x