dwright37/llm-knowledge-collapse

---
dataset_info:
- config_name: clusters
features:
- name: group
dtype: string
- name: topic
dtype: string
- name: factoid
dtype: string
- name: model_id
dtype: string
- name: prompt_index
dtype: int64
- name: setting
dtype: string
- name: cluster
dtype: int64
splits:
- name: clusters
num_bytes: 6373554945
num_examples: 69921477
download_size: 3084071661
dataset_size: 6373554945
- config_name: full_responses
features:
- name: text
dtype: string
- name: topic_id
dtype: int64
- name: user_prompt
dtype: string
- name: model_id
dtype: string
- name: topic
dtype: string
- name: prompt_index
dtype: int64
- name: setting
dtype: string
splits:
- name: full_responses
num_bytes: 8612894870
num_examples: 1581000
download_size: 4137238493
dataset_size: 8612894870
- config_name: templates
features:
- name: templates
dtype: string
splits:
- name: train
num_bytes: 14404
num_examples: 200
download_size: 8463
dataset_size: 14404
- config_name: topics
features:
- name: topic
dtype: string
- name: country
dtype: string
splits:
- name: train
num_bytes: 4982
num_examples: 155
download_size: 4467
dataset_size: 4982
configs:
- config_name: clusters
data_files:
- split: clusters
path: clusters/clusters-*
- config_name: full_responses
data_files:
- split: full_responses
path: full_responses/full_responses-*
- config_name: templates
data_files:
- split: train
path: templates/train-*
- config_name: topics
data_files:
- split: train
path: topics/train-*
---

# "Epistemic Diversity and Knowledge Collapse in Large Language Models" [(Wright et al. 2025)](https://arxiv.org/pdf/2510.04226)

[](https://arxiv.org/pdf/2510.04226)   [](https://github.com/dwright37/llm-knowledge)   [](https://pypi.org/project/llm-knowledge/)

Authors: Dustin Wright, Sarah Masud, Jared Moore, Srishti Yadav, Maria Antoniak, Peter Ebert Christiensen, Chan Young Park, and Isabelle Augenstein

Contains all 1.6M responses and 70M claims used to measure LLM epistemic diversity in the paper "Epistemic Diversity and Knowledge Collapse in Large Language Models" [(Wright et al. 2025)](https://arxiv.org/pdf/2510.04226)

```
@article{wright2025epistemicdiversity,
title={Epistemic Diversity and Knowledge Collapse in Large Language Models},
author={Dustin Wright and Sarah Masud and Jared Moore and Srishti Yadav
and Maria Antoniak and Chan Young Park and Isabelle Augenstein},
year={2025},
journal={arXiv preprint arXiv:2510.04226},
}
```

## Dataset Details
The data is generated by prompting 27 instruction fine-tuned LLMs in both RAG and non-RAG settings to generate responses to 155 different topics with 200 prompt variations.
These responses are then decomposed into individual claims, which are further clustered together using natural language inference in order to group the claims into clusters
of equivalent meaning.

The dataset contains four subsets: `full_reponses`, `clusters`, `topics` and `templates`. The `full_responses` subset contains the original 1.6M full responses from all 27 LLMs, for all 155 topics, and with all 200 prompt variations.
The `clusters` subset contains the 70M decomposed claims from the original full responses and their cluster IDs indicating which claims belong to the same meaning class. The `topics
subset contains just the 155 topics and the countries that they are affiliated with. The `templates` subset contains the 200 prompt tempaltes used in the study.

**NOTE**: If you use either the topics or the prompt templates, you should also cite the following paper where 30 topics and all of the prompt templates were sourced:

```
@misc{röttger2025issuebenchmillionsrealisticprompts,
title={IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance},
author={Paul Röttger and Musashi Hinck and Valentin Hofmann and Kobi Hackenburg and Valentina Pyatkin and Faeze Brahman and Dirk Hovy},
year={2025},
eprint={2502.08395},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2502.08395},
}
```

### Dataset Description

<!-- Provide a longer summary of what this dataset is. -->

The data in the `clusters` subset have the following columns:

- `group`: A group ID indicating which topic the claim is about.

- `topic`: The name of the topic for this group

- `factoid`: An individual claim derived from a response.

- `model_id`: The ID of the model that generated the claim.

- `prompt_index`: An index to the prompt variation that generated this claim. Can be used to match the claim to the original response in the `full_responses` subset.

- `setting`: The generation setting (either `ift` for instruction fine-tuned or `rag` for RAG)

- `cluster`: The cluster ID which this claim belongs to. Note that the cluster IDs are shared within a given topic only, and are shared across all models, settings, and prompts.

The data in the `full_responses` subset have the following columns:

- `text`: The full text response for this model on this particular prompt

- `topic_id`: An ID for the topic in this response.

- `user_prompt`: The prompt used to generate this response

- `model_id`: The model used to generate this response

- `topic`: The topic of this response

- `prompt_index`: An index to the prompt variation that generated this claim

- `setting`: The generation setting (either `ift` for instruction fine-tuned or `rag` for RAG)

The data in the `topic` subset have the following columns:

- `topic`: The topic name

- `country`: The country associated with this topic

The data in `templates` have a single column containing each prompt template.

## Additional Info

- **Curated by:** The authors (see above)
- **Language(s) (NLP):** English
- **License:** MIT

## Citation
```
@article{wright2025epistemicdiversity,
title={Epistemic Diversity and Knowledge Collapse in Large Language Models},
author={Dustin Wright and Sarah Masud and Jared Moore and Srishti Yadav
and Maria Antoniak and Chan Young Park and Isabelle Augenstein},
year={2025},
journal={arXiv preprint arXiv:2510.04226},
}
```

## Dataset Card Authors

Dustin Wright

## Dataset Card Contact

[Dustin Wright](https://dustinbwright.com)

数据集信息:
- 配置名称: clusters
特征列:
- 列名: group, 数据类型: 字符串(string)
- 列名: topic, 数据类型: 字符串(string)
- 列名: factoid, 数据类型: 字符串(string)
- 列名: model_id, 数据类型: 字符串(string)
- 列名: prompt_index, 数据类型: 64位整数(int64)
- 列名: setting, 数据类型: 字符串(string)
- 列名: cluster, 数据类型: 64位整数(int64)
数据集划分:
- 划分名称: clusters, 字节大小: 6373554945, 样本数量: 69921477
下载大小: 3084071661, 数据集总大小: 6373554945
- 配置名称: full_responses
特征列:
- 列名: text, 数据类型: 字符串(string)
- 列名: topic_id, 数据类型: 64位整数(int64)
- 列名: user_prompt, 数据类型: 字符串(string)
- 列名: model_id, 数据类型: 字符串(string)
- 列名: topic, 数据类型: 字符串(string)
- 列名: prompt_index, 数据类型: 64位整数(int64)
- 列名: setting, 数据类型: 字符串(string)
数据集划分:
- 划分名称: full_responses, 字节大小: 8612894870, 样本数量: 1581000
下载大小: 4137238493, 数据集总大小: 8612894870
- 配置名称: templates
特征列:
- 列名: templates, 数据类型: 字符串(string)
数据集划分:
- 划分名称: train, 字节大小: 14404, 样本数量: 200
下载大小: 8463, 数据集总大小: 14404
- 配置名称: topics
特征列:
- 列名: topic, 数据类型: 字符串(string)
- 列名: country, 数据类型: 字符串(string)
数据集划分:
- 划分名称: train, 字节大小: 4982, 样本数量: 155
下载大小: 4467, 数据集总大小: 4982
配置项:
- 配置名称: clusters, 数据文件:
- 划分: clusters, 路径: clusters/clusters-*
- 配置名称: full_responses, 数据文件:
- 划分: full_responses, 路径: full_responses/full_responses-*
- 配置名称: templates, 数据文件:
- 划分: train, 路径: templates/train-*
- 配置名称: topics, 数据文件:
- 划分: train, 路径: topics/train-*

# 《大语言模型中的认知多样性与知识坍缩》[(Wright等人，2025)](https://arxiv.org/pdf/2510.04226)

[](https://arxiv.org/pdf/2510.04226) &nbsp; [](https://github.com/dwright37/llm-knowledge) &nbsp; [](https://pypi.org/project/llm-knowledge/)

作者: Dustin Wright, Sarah Masud, Jared Moore, Srishti Yadav, Maria Antoniak, Peter Ebert Christiensen, Chan Young Park, Isabelle Augenstein

本数据集包含论文《大语言模型中的认知多样性与知识坍缩》[(Wright等人，2025)](https://arxiv.org/pdf/2510.04226)中用于衡量大语言模型认知多样性的全部160万条响应与7000万条断言。

@article{wright2025epistemicdiversity,
title={Epistemic Diversity and Knowledge Collapse in Large Language Models},
author={Dustin Wright and Sarah Masud and Jared Moore and Srishti Yadav
and Maria Antoniak and Chan Young Park and Isabelle Augenstein},
year={2025},
journal={arXiv preprint arXiv:2510.04226},
}

## 数据集详情
本数据集通过在检索增强生成（Retrieval-Augmented Generation，RAG）与非RAG两种设置下，对27个经过指令微调的大语言模型（Large Language Model，LLM）进行提示，使其针对155个不同主题生成200种提示变体的响应。随后将这些响应拆解为单条事实性断言，并通过自然语言推理完成聚类，将语义等价的断言归入同一聚类簇中。

本数据集包含四个子集：`full_reponses`、`clusters`、`topics`与`templates`。其中`full_responses`子集包含来自全部27个大语言模型、针对全部155个主题以及全部200种提示变体的原始160万条完整响应。`clusters`子集包含来自原始完整响应的7000万条拆解后的事实性断言，以及对应的聚类簇ID，用于标识同语义类别的断言。`topics`子集仅包含155个主题及其所属国家。`templates`子集包含本研究中使用的200条提示模板。

**注意**：若使用主题或提示模板，需同时引用以下论文，其中包含了30个主题与全部提示模板的来源：

@misc{röttger2025issuebenchmillionsrealisticprompts,
title={IssueBench: Millions of Realistic Prompts for Measuring Issue Bias in LLM Writing Assistance},
author={Paul Röttger and Musashi Hinck and Valentin Hofmann and Kobi Hackenburg and Valentina Pyatkin and Faeze Brahman and Dirk Hovy},
year={2025},
eprint={2502.08395},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2502.08395},
}

## 数据集描述
<!-- 请提供该数据集的详细摘要。 -->

`clusters`子集的数据包含以下列：
- `group`：主题分组ID，用于标识该断言所属的主题。
- `topic`：该分组对应的主题名称。
- `factoid`：从响应中拆解得到的单条事实性断言。
- `model_id`：生成该断言的大语言模型ID。
- `prompt_index`：生成该断言所使用的提示变体索引，可用于匹配`full_responses`子集中的原始响应。
- `setting`：生成设置，取值为`ift`（指令微调）或`rag`（检索增强生成）。
- `cluster`：该断言所属的聚类簇ID。请注意，聚类簇ID仅在同一主题内有效，且跨所有模型、设置与提示变体共享。

`full_responses`子集的数据包含以下列：
- `text`：该模型针对该特定提示生成的完整响应文本。
- `topic_id`：该响应所属主题的ID。
- `user_prompt`：用于生成该响应的提示文本。
- `model_id`：生成该响应的大语言模型ID。
- `topic`：该响应对应的主题名称。
- `prompt_index`：生成该响应所使用的提示变体索引。
- `setting`：生成设置，取值为`ift`（指令微调）或`rag`（检索增强生成）。

`topics`子集的数据包含以下列：
- `topic`：主题名称。
- `country`：该主题关联的国家。

`templates`子集仅包含单个列，存储全部提示模板。

## 附加信息
- **整理方**：上述作者
- **NLP所用语言**：英语
- **许可证**：MIT协议

## 引用

@article{wright2025epistemicdiversity,
title={Epistemic Diversity and Knowledge Collapse in Large Language Models},
author={Dustin Wright and Sarah Masud and Jared Moore and Srishti Yadav
and Maria Antoniak and Chan Young Park and Isabelle Augenstein},
year={2025},
journal={arXiv preprint arXiv:2510.04226},
}

## 数据集卡片作者
Dustin Wright

## 数据集卡片联系方式
[Dustin Wright](https://dustinbwright.com)