Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas

This repository contains the datasets and materials used to analyze and replicate the results presented in our paper investigating how persona-based prompting affects the political orientations of Large Language Models (LLMs). Contents The repository includes files organized by model (Mistral, Llama, Qwen, and Zephyr) and experimental condition (base, right-authoritarian [ra], and left-libertarian [ll]): Model Response Data *_persona_compass_base.pqt: Political compass test responses for each model using baseline persona descriptions *_persona_compass_ra.pqt: Responses after injecting right-authoritarian descriptors *_persona_compass_ll.pqt: Responses after injecting left-libertarian descriptors Configuration and Input Files personas.json: Collection of synthetic persona descriptions from PersonaHub used in the experiments token_personas.json: Tokenized versions of the persona descriptions political_compass_statements.json: The 62 statements from the Political Compass Test used for evaluation prompts.json: Prompt templates used for model interactions baseLLMsPoliticalView.json: Default political orientations of the models without persona prompting Related Code Repository The code used to analyze this data and reproduce the results presented in the paper can be found at: https://github.com/d-lab/llm-political-personas File Placement Instructions After downloading, organize the files as follows: Configuration and Input Files Place all the configuration files in the data/raw/ directory. Model Response Files Rename all model-specific .pqt files to persona_compass.pqt and place them in their respective directories: Base condition files: data/processed/Llama-3.1-8B-Instruct/base/persona_compass.pqt data/processed/Mistral-7B-Instruct-v0.3/base/persona_compass.pqt data/processed/Qwen2.5-7B-Instruct/base/persona_compass.pqt data/processed/zephyr-7b-beta/base/persona_compass.pqt Right-authoritarian condition files: data/processed/Llama-3.1-8B-Instruct/right_authoritarian_personas/persona_compass.pqt data/processed/Mistral-7B-Instruct-v0.3/right_authoritarian_personas/persona_compass.pqt data/processed/Qwen2.5-7B-Instruct/right_authoritarian_personas/persona_compass.pqt data/processed/zephyr-7b-beta/right_authoritarian_personas/persona_compass.pqt Left-libertarian condition files: data/processed/Llama-3.1-8B-Instruct/left_libertarian_personas/persona_compass.pqt data/processed/Mistral-7B-Instruct-v0.3/left_libertarian_personas/persona_compass.pqt data/processed/Qwen2.5-7B-Instruct/left_libertarian_personas/persona_compass.pqt data/processed/zephyr-7b-beta/left_libertarian_personas/persona_compass.pqt