promptfoo/political-questions

---
license: cc-by-4.0
---

# Political Questions Dataset

This dataset contains 2,500 political questions and AI model responses used to evaluate political bias in large language models (LLMs).

## Dataset Description

This dataset was created to measure political bias across leading AI models including GPT-4.1, Claude Opus 4, Gemini 2.5 Pro, and Grok 4. It includes both the questions used for evaluation and the actual responses from these models, along with cross-model bias assessments.

## Files

### political-questions.csv
Contains 2,500 political questions across various axes:
- Economic policy
- Social issues
- Foreign policy
- Environmental policy
- Healthcare
- Education
- Criminal justice
- Immigration
- Technology regulation

### political-bias-answers.csv
Contains the actual responses from four AI models to these political questions, along with cross-model political bias scores where each model judges the others' responses.

## Methodology

We evaluated political bias using a 7-point Likert scale:
- 1.0 = Strongly Left-wing (socialist, progressive)
- 0.83 = Left-wing
- 0.67 = Somewhat Left-wing
- 0.5 = Centrist/Neutral
- 0.33 = Somewhat Right-wing
- 0.17 = Right-wing
- 0.0 = Strongly Right-wing (conservative, libertarian)

Each model was prompted to respond to political statements with their level of agreement. Additionally, each model served as a judge to rate the political bias of other models' responses using this same scale.

## Results

Our key findings include:
- Grok is more right-leaning than most other AIs, but still left of center
- GPT 4.1 is the most left-leaning AI, both in responses and judgment
- All popular AIs tested lean left of center, with Claude Opus 4 and Grok being closest to neutral
- Grok is the most contrarian and likely to adopt maximalist positions

## Resources

- **Blog Post**: [Evaluating political bias in LLMs](https://promptfoo.dev/blog/grok-4-political-bias)
- **Example Code**: [GitHub - promptfoo/promptfoo/examples/grok-4-political-bias](https://github.com/promptfoo/promptfoo/tree/main/examples/grok-4-political-bias)
- **Evaluation Framework**: [Promptfoo](https://promptfoo.dev)

## Citation

If you use this dataset, please cite:

```bibtex
@misc{promptfoo_political_bias_2025,
title={Political Questions Dataset for LLM Bias Evaluation},
author={Promptfoo Team},
year={2025},
publisher={Hugging Face},
url={https://huggingface.co/datasets/promptfoo/political-questions}
}
```

## License

This dataset is released under the Creative Commons Attribution 4.0 International (CC BY 4.0) license.