Dataset for "Do AIs pass the Ideological Turing Test?"

A growing empirical literature reports that large language models (LLMs) exhibit a left-of-center political tilt. Much of this work, however, measures viewpoint affinity. That is, which ideological side a model appears to favor, and treats affinity itself as evidence of bias. The Viewpoint Preferences–Epistemic Integrity (VPEI) framework argues that this conflation is mistaken: political bias requires epistemic error, not merely directional preference. One component of epistemic integrity is epistemic emulation: accuracy in perspective-taking. Its canonical test is the Ideological Turing Test (ITT), which asks whether a system can articulate an argument from a given ideological perspective in a way that adherents of that perspective judge to accurately represent their own views. Here, we characterize how six state-of-the-art LLMs perform on the ITT. In Study 1, the models generated the strongest possible left- and right-wing arguments across 15 contested U.S. political topics. A sample of 408 U.S. respondents then rated how well each argument represented their own views. Left-of-center respondents rated the models’ arguments as more representative of their views than right-of-center respondents did. This asymmetry is not conclusive of model epistemic failure as it could reflect either weaker model performance in articulating right-of-center perspectives or greater evaluative stringency among right-of-center respondents. To complement the first study, Study 2 tested whether models accurately estimate what different ideological and partisan groups believe. Thus, we asked the models to estimate the full response distributions of liberals, conservatives, Democrats, and Republicans on real 2024 American National Election Studies survey items and scored their estimates against survey-weighted ground truth. Across both studies, the models performed well for both sides but estimated left-of-center beliefs more accurately than right-of-center beliefs. This pattern was directionally consistent across models but modest in magnitude, suggesting a mild asymmetry in perspective-taking accuracy rather than a wholesale failure to represent the political right.