Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection

Full-Text PDF<br> Title: Are Neural Language Models Good Plagiarists? A Benchmark for Neural Paraphrase Detection<br> Authors: Jan Philip Wahle, Terry Ruas, Norman Meuschke, and Bela Gipp<br> Contact email: [e-mail]; [e-mail]<br> Venue: JCDL<br> Year: 2021<br> ================================================================<br> <strong>Dataset Description:</strong> <em><strong>Training:</strong></em><br> 1,474,230 aligned paragraphs (98,282 original, 1,375,948 paraphrased with 3 models and 5 hyperparameter configurations each 98,282) extracted from 4,012 (English) Wikipedia articles. <em><strong>Testing:</strong></em><br> BERT-large (cased): <br> arXiv - Original - 20,966; Paraphrased - 20,966; <br> Theses - Original - 5,226; Paraphrased - 5,226;<br> Wikipedia - Original - 39,241; Paraphrased - 39,241;<br> <br> RoBERTa-large (cased): <br> arXiv - Original - 20,966; Paraphrased - 20,966; <br> Theses - Original - 5,226; Paraphrased - 5,226;<br> Wikipedia - Original - 39,241; Paraphrased - 39,241; Longformer-large (uncased): <br> arXiv - Original - 20,966; Paraphrased - 20,966; <br> Theses - Original - 5,226; Paraphrased - 5,226;<br> Wikipedia - Original - 39,241; Paraphrased - 39,241; ================================================================ Dataset Structure: <strong>[og]</strong> folder: original. The original documents are split by the data source with the following folders: <strong>[arxiv]</strong> <strong>[thesis]</strong> <strong>[wikipedia]</strong> <strong>[wikipedia_train]</strong> <strong>[`model_name`_mlm_prob_`probability`] (e.g., bert-large-cased_mlm_prob_0.15): </strong>contains all paraphrased examples using the model with name `model_name` and Masked Language Modeling probability `probability`.<br> Each paraphrase model/probability folder contains the corresponding paraphrased documents according to<strong> [of]</strong>: <strong>[arxiv]</strong> <strong>[thesis]</strong> <strong>[wikipedia]</strong> <strong>[wikipedia_train]</strong> hparams.yml hparams.yml contains the hyperparameters to reconstruct the dataset using the official repository. ================================================================ Files:<br> On the lowest folder level, each `.txt` file contains exactly one paragraph. The filename contains either "ORIG" for original, or "SPUN" for paraphrased. ================================================================ Code:<br> To avoid misuse of the code for constructing machine-paraphrased plagiarism, you must consent to our Terms and Conditions and send the signed version via mail to one of the contact addresses above to obtain access to our repository (see TermsAndConditions.pdf).

全文PDF<br>标题：神经语言模型是否擅长充当抄袭者？面向神经释义检测的基准数据集<br>作者：扬·菲利普·瓦勒（Jan Philip Wahle）、特里·鲁阿斯（Terry Ruas）、诺曼·莫伊施克（Norman Meuschke）、贝拉·吉普（Bela Gipp）<br>联系邮箱：[e-mail]; [e-mail]<br>发表会议：JCDL<br>年份：2021<br>================================================================<br><strong>数据集描述：</strong><br><em><strong>训练集：</strong></em><br>共1,474,230条对齐段落（其中原始段落98,282条，经3种模型、每种模型搭配5种超参数（hyperparameter）配置生成的释义段落共1,375,948条，单种模型单配置对应98,282条释义段落），数据从4012篇（英语）维基百科文章中提取得到。<br><em><strong>测试集：</strong></em><br>BERT-large（大小写敏感）：<br>arXiv数据集：原始文本20,966条，释义文本20,966条；<br>学位论文数据集：原始文本5,226条，释义文本5,226条；<br>维基百科数据集：原始文本39,241条，释义文本39,241条；<br><br>RoBERTa-large（大小写敏感）：<br>arXiv数据集：原始文本20,966条，释义文本20,966条；<br>学位论文数据集：原始文本5,226条，释义文本5,226条；<br>维基百科数据集：原始文本39,241条，释义文本39,241条；<br>Longformer-large（大小写不敏感）：<br>arXiv数据集：原始文本20,966条，释义文本20,966条；<br>学位论文数据集：原始文本5,226条，释义文本5,226条；<br>维基百科数据集：原始文本39,241条，释义文本39,241条；<br>================================================================<br><strong>数据集结构：</strong><br><strong>[og]</strong> 文件夹：存放原始文本。原始文本按数据源划分为以下子文件夹：<strong>[arxiv]</strong>、<strong>[thesis]</strong>、<strong>[wikipedia]</strong>、<strong>[wikipedia_train]</strong>。<br><strong>[`model_name`_mlm_prob_`probability`]</strong>（例如：bert-large-cased_mlm_prob_0.15）文件夹：存放使用指定模型`model_name`与掩码语言模型（Masked Language Model, MLM）概率`probability`生成的全部释义样本。每个释义模型/超参数配置文件夹下，同样按照数据源划分为<strong>[arxiv]</strong>、<strong>[thesis]</strong>、<strong>[wikipedia]</strong>、<strong>[wikipedia_train]</strong>子文件夹。<br>根目录下的`hparams.yml`文件包含了通过官方仓库复现该数据集所需的超参数配置。<br>================================================================<br><strong>文件说明：</strong><br>在最低层级的文件夹中，每个`.txt`文件恰好包含一个段落。文件名中若包含"ORIG"则代表原始文本，包含"SPUN"则代表释义文本。<br>================================================================<br><strong>代码获取说明：</strong><br>为避免该代码被滥用于生成机器释义式抄袭内容，您需同意本研究的条款与条件，并将签署后的版本通过邮件发送至上述任一联系地址，以获取仓库访问权限（详见TermsAndConditions.pdf）