chongli17/AG-Pair

AG-Pair数据集是基于AGs News原始数据集构建的，包含120k个训练样本，涵盖四个主题。给定一对新闻作为输入，模型需要预测它们是否属于同一主题（相同）或不同主题（不同）。为了生成该数据集，AGs News中的样本以随机顺序迭代，并以相等的机会将同一主题或不同主题的样本组合在一起。因此，两个类别的训练样本数量均为60k。此外，AGs News中的每个新闻在AG-Pair数据集中出现两次，以保持相同的词频。The AG-Pair dataset is constructed based on the original AG's News dataset, which contains 120k training samples covering four topics. Given a pair of news articles as input, the model is required to predict whether the two articles belong to the same topic (denoted as "same") or different topics (denoted as "different"). To generate this dataset, samples from AG's News are iterated in random order, and paired with samples of the same topic or different topics with equal probability. As a result, the number of training samples for each of the two categories is 60k. Additionally, each news article in the original AG's News dataset appears exactly twice in the AG-Pair dataset to maintain the same word frequency.

原始信息汇总

数据集概述

数据集名称

AG-Pair 数据集

数据集构建

AG-Pair 数据集是从原始的 AGs News 数据集构建而来，该原始数据集包含来自四个主题的 120,000 个训练样本。该数据集的任务是给定一对新闻作为输入，模型需要预测它们是否属于同一主题（Same）或不同主题（Different）。

数据集生成方法

在生成 AG-Pair 数据集时，样本在 AGs News 中以随机顺序迭代，并且有相等的机会与同一主题或其他三个主题的样本组合。因此，两个类别的训练样本数量均为 60,000 个。此外，AGs News 中的每条新闻在 AG-Pair 数据集中恰好出现两次，以保持相同的词频。

数据集信息

• 任务类别: 文本分类
• 语言: 英语
• 大小类别: 100K<n<1M
• 许可证: unknown

数据集策展人

• Chong Li ([e-mail])

引用信息

@inproceedings{li-etal-2023-FunctionalSpecialization, author = {Chong Li and Shaonan Wang and Yunhao Zhang and Jiajun Zhang and Chengqing Zong}, title = "Interpreting and Exploiting Functional Specialization in Multi-Head Attention under Multi-task Learning", booktitle = "Proceedings of the 2023 Conference on Empirical Methods in Natural Language Processing", year = "2023", address = "Singapore", publisher = "Association for Computational Linguistics", }