Twitter News URL Corpus

Twitter News URL Corpus是由俄亥俄州立大学计算机科学与工程系的研究团队创建的一个持续增长的数据集，包含51,524对句子，主要从Twitter上通过共享URL链接的推文收集而来。该数据集的特点是简单高效，无需分类器或人工干预即可大量收集句子对，适用于下游NLP任务，如短语释义提取。数据集每月自动标记新增超过30,000对句子，精度约70%，旨在解决释义研究中缺乏平行语料的问题，为自动释义识别提供首个跨领域基准。

The Twitter News URL Corpus is a continuously expanding dataset developed by a research team from the Department of Computer Science and Engineering at The Ohio State University, consisting of 51,524 sentence pairs primarily collected from tweets sharing URL links on Twitter. Boasting simplicity and efficiency, this dataset enables large-scale collection of sentence pairs without the need for classifiers or manual intervention, making it suitable for downstream natural language processing (NLP) tasks such as phrase paraphrase extraction. The dataset automatically annotates over 30,000 new sentence pairs each month, with an annotation accuracy of approximately 70%. It aims to address the shortage of parallel corpora in paraphrase research and serves as the first cross-domain benchmark for automatic paraphrase identification.

搜集汇总

背景与挑战

背景概述

Twitter News URL Corpus是一个由俄亥俄州立大学创建的持续增长数据集，包含51,524对从Twitter推文通过共享URL收集的句子对，特点是无需分类器或人工干预即可高效收集，适用于NLP任务如短语释义提取。它每月自动标记新增超过30,000对句子，精度约70%，旨在解决释义研究中平行语料不足的问题，为自动释义识别提供跨领域基准。