Authoritarian AI: Evaluating Cross-model Variability in Left and Right-wing Authoritarianism

Abstract
Accurately representing the diversity of human thought, while simultaneously avoiding political extremism and bias, remains a major hurdle for the responsible use of commercial LLMs. We systematically evaluated the biases and boundaries of authoritarianism in three popular commercial LLMs (Grok, Claude, ChatPGT) using Left-wing and Right-wing Political Authoritarianism (LWA/RWA), psychological constructs designed to capture the essence of authoritarianism and that predict problematic attitudes and behaviours. Using extreme left and right-wing political personas, we systematically probe the biases and boundaries of political authoritarianism in popular commercial LLMs and directly compare this variance to a large human sample. We show that popular models both systematically over- and under-express authoritarianism relative to human populations. Models appear to caricature levels of polarization between conservatives and liberals, underestimating extreme-right agreement with LWA scale items, and grossly overestimating levels of RWA among the extreme-right. All models demonstrated levels of RWA commensurate with political extremists in certain contexts, suggesting commercial LLM’s potentially exacerbate ideological polarization This highlights a critical challenge for Responsible AI (RAI) and demonstrates the need for a toolkit of psychometric tests to audit LLM biases, not only for their theoretical validity, but for their ability to link LLM outputs to downstream attitudes and behaviours and ensure models align with diverse human values.