Paper information in the topic of large language models

This dataset supports the findings in the preprint 'Academic collaboration on large language model studies increases overall but varies across disciplines.' The study aims to explore the application of large language models (LLMs) in scientific disciplines and their implications for interdisciplinary collaboration.

To build LLM paper group, we start with a broad search using general terms related to LLMs and popular models based on the MMLU benchmark spanning from October 2018 to September 2024. We apply this search to the title and abstract to avoid excessive noise in the dataset and then undergo a series of filtering stepsto enhance relevance and remove duplicates. The resulting dataset contains 59,293 papers.

In addition to the paper group in the topic of LLMs, we establish two control groups. The first control group focuses on machine learning (ML) papers. We select ML as a control because it is a well-established field from which LLM emerged as a subfield. To construct this group, we collect a random sampling of 70,945 papers containing the phrase ''machine learning'' in either their title or abstract. To provide an even broader perspective beyond AI-related fields, we create a second control group consisting of a random sample of 73,110 papers from all other research categories---specifically, papers that belong neither to the ML nor LLM categories. 

The three files below contain the cleaned samples collected from OpenAlex, which are derived from the original files. 

LLM: llm-cleaned-samples.csv

ML: ml-cleaned-samples.csv

Non-LLM/ML: non-llm-cleaned-samples.csv

The three zip files below contain author affiliation information (including departmental discipline) extracted by GPT-4o-mini to support the departmental analysis in the paper:

LLM: llm-author-affiliations.zip

ML: ml-author-affiliations.zip

Non-LLM/ML: non-llm-author-affiliations.zip

The three files below contain the paper information used to support all the analysis in our paper:

LLM: llm-information-entropy.csv

ML: ml-information-entropy.csv

Non-LLM/ML: non-llm-information-entropy.csv

If you have any additional questions, please feel free to contact [e-mail] or [e-mail].

本数据集支撑预印本论文《学术合作在大语言模型研究中整体增长但跨学科差异显著》中的研究结论。该研究旨在探索大语言模型（Large Language Model, LLM）在各科学学科中的应用，及其对跨学科合作的影响。

为构建大语言模型论文组，我们首先以与大语言模型及基于MMLU基准测试的主流模型相关的通用术语为检索词，开展广谱检索，检索时间范围为2018年10月至2024年9月。我们将检索范围限定在论文标题与摘要中，以避免数据集引入过多噪声，随后通过一系列筛选步骤提升相关性并去除重复文献。最终得到的数据集共包含59293篇论文。

除上述大语言模型主题论文组外，我们还设立了两个对照组。第一对照组聚焦机器学习（Machine Learning, ML）论文。之所以选择机器学习作为对照，是因为该领域发展成熟，而大语言模型正是从该领域衍生出的子领域。为构建该对照组，我们随机抽取了70945篇标题或摘要中包含“机器学习”短语的论文。为提供超出人工智能相关领域的更广泛视角，我们设立了第二对照组：从所有其他研究类别（即既不属于机器学习也不属于大语言模型类别的论文）中随机抽取73110篇论文作为样本。

以下三个文件为从OpenAlex获取并经清洗后的样本数据，均源自原始文件：
- 大语言模型组：llm-cleaned-samples.csv
- 机器学习组：ml-cleaned-samples.csv
- 非大语言模型/机器学习组：non-llm-cleaned-samples.csv

以下三个压缩文件包含由GPT-4o-mini提取的作者所属机构信息（包含院系学科），用于支撑论文中的学科分析：
- 大语言模型组：llm-author-affiliations.zip
- 机器学习组：ml-author-affiliations.zip
- 非大语言模型/机器学习组：non-llm-author-affiliations.zip

以下三个文件包含用于支撑本论文所有分析的论文信息：
- 大语言模型组：llm-information-entropy.csv
- 机器学习组：ml-information-entropy.csv
- 非大语言模型/机器学习组：non-llm-information-entropy.csv

若您有任何额外问题，请联系[e-mail]或[e-mail]。