Data_Sheet_2_Transitivity Violations Undermine Rating Scales in Motivation Research.PDF

Measures of psychological attributes, such as motivation, typically involve rating scales, assuming that an attribute can be ordered. If an attribute has an ordinal structure, its levels stand in ordinal relations to one another, and these must be transitive. We tested if transitivity is preserved when people compare different motives in terms of their importance to learning. We found transitivity violations in both strict (Study 1) and non-strict (Study 2) orderings in about half of the participants. Nevertheless, based on the distribution of such violations, we conclude that an ordinal structure of motivation can be found, but only when levels of motives differ noticeably. As the levels become subjectively similar, transitivity is not preserved, and the ordinal structure cannot be justified even in non-strict ordering. The findings question the mainstream practice of measuring psychological attributes before their structure is properly explored.