Mapping and Influencing the Political Ideology of Large Language Models using Synthetic Personas

This repository contains the datasets and materials used to analyze and replicate the results presented in our paper investigating how persona-based prompting affects the political orientations of Large Language Models (LLMs).

Contents

The repository includes files organized by model (Mistral, Llama, Qwen, and Zephyr) and experimental condition (base, right-authoritarian [ra], and left-libertarian [ll]):

Model Response Data

*_persona_compass_base.pqt: Political compass test responses for each model using baseline persona descriptions

*_persona_compass_ra.pqt: Responses after injecting right-authoritarian descriptors

*_persona_compass_ll.pqt: Responses after injecting left-libertarian descriptors

Configuration and Input Files

personas.json: Collection of synthetic persona descriptions from PersonaHub used in the experiments

token_personas.json: Tokenized versions of the persona descriptions

political_compass_statements.json: The 62 statements from the Political Compass Test used for evaluation

prompts.json: Prompt templates used for model interactions

baseLLMsPoliticalView.json: Default political orientations of the models without persona prompting

Related Code Repository

The code used to analyze this data and reproduce the results presented in the paper can be found at: https://github.com/d-lab/llm-political-personas

File Placement Instructions

After downloading, organize the files as follows:

Configuration and Input Files

Place all the configuration files in the data/raw/ directory.

Model Response Files

Rename all model-specific .pqt files to persona_compass.pqt and place them in their respective directories:

Base condition files:

data/processed/Llama-3.1-8B-Instruct/base/persona_compass.pqt

data/processed/Mistral-7B-Instruct-v0.3/base/persona_compass.pqt

data/processed/Qwen2.5-7B-Instruct/base/persona_compass.pqt

data/processed/zephyr-7b-beta/base/persona_compass.pqt

Right-authoritarian condition files:

data/processed/Llama-3.1-8B-Instruct/right_authoritarian_personas/persona_compass.pqt

data/processed/Mistral-7B-Instruct-v0.3/right_authoritarian_personas/persona_compass.pqt

data/processed/Qwen2.5-7B-Instruct/right_authoritarian_personas/persona_compass.pqt

data/processed/zephyr-7b-beta/right_authoritarian_personas/persona_compass.pqt

Left-libertarian condition files:

data/processed/Llama-3.1-8B-Instruct/left_libertarian_personas/persona_compass.pqt

data/processed/Mistral-7B-Instruct-v0.3/left_libertarian_personas/persona_compass.pqt

data/processed/Qwen2.5-7B-Instruct/left_libertarian_personas/persona_compass.pqt

data/processed/zephyr-7b-beta/left_libertarian_personas/persona_compass.pqt