Replication Data for: ANES Scales Often Don’t Measure What You Think They Measure

Political surveys often include multi-item scales to measure individual predispositions such as authoritarianism, egalitarianism, or racial resentment. Scholars use these scales to examine group differences in these predispositions, comparing women to men, rich to poor, or Republicans to Democrats. Such research implicitly assumes that, say, Republicans' and Democrats' responses to the egalitarianism scale measure the same construct in the same metric. This research rarely evaluates whether the data possess the characteristics necessary to justify this equivalence assumption. We present a framework to test this assumption and correct scales when it fails to hold. Examining 13 commonly used scales on the 2012 and 2016 ANES, we find widespread violations of the equivalence assumption. These violations often bias the estimated magnitude or direction of theoretically important group differences. These results suggest we must reevaluate what we think we know about the causes and consequences of authoritarianism, egalitarianism, and other predispositions.

政治调查通常会采用多项目量表（multi-item scales）来测量个体倾向性，例如威权主义（authoritarianism）、平等主义（egalitarianism）与种族怨恨（racial resentment）。学者们借助此类量表考察这类倾向性的群体差异，例如对比女性与男性、富裕群体与贫困群体，或是共和党人与民主党人之间的不同。此类研究默认，譬如共和党人与民主党人在平等主义量表上的作答，所测量的是同一构念（construct）且处于同一测量量尺之下。但这类研究极少会评估数据是否具备验证该等价性假设所需的特征。我们提出了一套用于检验该假设的框架，并可在假设不成立时对量表进行校正。通过分析2012年与2016年美国国家选举研究（American National Election Studies, ANES）中的13个常用量表，我们发现等价性假设被广泛违反。这类违反情况往往会使具有理论重要性的群体差异的估计值在幅度或方向上产生偏差。上述结果表明，我们必须重新审视当前对威权主义、平等主义及其他倾向性的成因与影响的既有认知。