election_questions

# Election Evaluations Dataset

## Dataset Summary

This dataset includes some of the evaluations we implemented to assess language models' ability to handle election-related information accurately, harmlessly, and without engaging in persuasion targeting.

## Dataset Description

The dataset consists of three CSV files, each focusing on a specific aspect of election-related evaluations:

1. `eu_accuracy_questions.csv`:
- Contains information-seeking questions about European Union (EU) election-related topics.
- The questions are designed to evaluate the accuracy of language models in providing factual information about EU elections.

2. `harmlessness_eval.csv`:
- Includes a mix of harmless and potentially harmful questions on a range of topics related to United States (US) elections.
- Each question is labeled as either "Harmless" or "Harmful."
- The goal is to assess whether language models can identify and refuse to answer questions that are labeled as "Harmful," while providing appropriate responses to "Harmless" questions.

3. `persuasion_targeting_eval.csv`:
- Contains questions related to various demographic groups within the United States in the context of elections.
- Each question is labeled as either "Harmless" or "Harmful."
- "Harmful" questions attempt to target specific populations for persuasion or manipulation.
- The objective is to evaluate whether language models can identify and refuse to answer questions labeled as "Harmful", while providing appropriate responses to "Harmless" questions.

## Disclaimer
Please note that these evaluations are generated by language models and may contain inaccuracies. While we conducted human evaluation on a subset of the dataset to assess quality, not all questions were manually reviewed. The dataset is intended for research purposes only and should not be considered a definitive assessment of language models' performance in handling election-related information.

## Usage
```python
from datasets import load_dataset
# Loading the data
dataset = load_dataset("Anthropic/election_questions")
```
## Contact
For questions, you can email esin at anthropic dot com

# 选举评估数据集

## 数据集概述
本数据集收录了我们为评估大语言模型（Large Language Model）准确、无害地处理选举相关信息，且不开展针对性劝服行为而实施的若干评估项目。

## 数据集说明
本数据集包含三个CSV文件，各自聚焦于选举相关评估的特定维度：
1. `eu_accuracy_questions.csv`：
- 收录了针对欧盟（European Union, EU）选举相关议题的信息查询类问题，旨在评估大语言模型在提供欧盟选举事实性信息时的准确性。
2. `harmlessness_eval.csv`：
- 涵盖了与美国（United States, US）选举相关的各类议题中，兼具无害与潜在有害属性的问题集合，每个问题均标注为“无害”或“有害”。
- 本评估的目标为检验大语言模型能否识别并拒绝回答标注为“有害”的问题，同时为“无害”问题提供恰当回复。
3. `persuasion_targeting_eval.csv`：
- 收录了选举场景下，与美国境内各类人口群体相关的问题，每个问题同样标注为“无害”或“有害”。
- 其中“有害”问题试图针对特定群体开展劝服或操纵行为，评估目标为检验大语言模型能否识别并拒绝回答“有害”类问题，同时为“无害”问题提供适配性回复。

## 免责声明
请注意，本评估内容由大语言模型生成，可能存在不准确之处。尽管我们对数据集的子集开展了人工评估以检验其质量，但并非所有问题均经过人工审核。本数据集仅用于研究目的，不得被视为对大语言模型处理选举相关信息能力的权威性评估。

## 使用方法
python
from datasets import load_dataset
# 加载数据集
dataset = load_dataset("Anthropic/election_questions")

## 联系方式
如有相关疑问，可发送邮件至 [e-mail]