Identifying Machine-Paraphrased Plagiarism

README.txt Title: <em>Identifying Machine-Paraphrased Plagiarism</em><br> Authors: Jan Philip Wahle, Terry Ruas, Tomas Foltynek, Norman Meuschke, and Bela Gipp<br> contact email: [e-mail]; [e-mail];<br> Venue: iConference<br> Year: 2022<br> ================================================================<br> <strong>Dataset Description:</strong> <em><strong>Training:</strong></em><br> 200,767 paragraphs (98,282 original, 102,485paraphrased) extracted from 8,024 Wikipedia (English) articles (4,012 original, 4,012 paraphrased using the SpinBot API). <em><strong>Testing:</strong></em><br> SpinBot: <br> arXiv - Original - 20,966; Spun - 20,867<br> Theses - Original - 5,226; Spun - 3,463<br> Wikipedia - Original - 39,241; Spun - 40,729<br> <br> SpinnerChief-4W: <br> arXiv - Original - 20,966; Spun - 21,671<br> Theses - Original - 2,379; Spun - 2,941<br> Wikipedia - Original - 39,241; Spun - 39,618<br> <br> SpinnerChief-2W: <br> arXiv - Original - 20,966; Spun - 21,719<br> Theses - Original - 2,379; Spun - 2,941<br> Wikipedia - Original - 39,241; Spun - 39,697 ================================================================<br> Dataset Structure: <strong>[human_evaluation]</strong> folder: human evaluation to identify human-generated text and machine-paraphrased text. It contains the files (original and spun) as for the answer-key for the survey performed with human subjects (all data is anonymous for privacy reasons). NNNNN.txt - whole document from which an extract was taken for human evaluation<br> key.txt.zip - information about each case (ORIG/SPUN)<br> results.xlsx - raw results downloaded from the survey tool (the extracts which humans judged are in the first line)<br> results-corrected.xlsx - at the very beginning, there was a mistake in one question (wrong extract). These results were excluded. <br> <strong>[automated_evaluation]: </strong>contains all files used for the automated evaluation considering [spinbot] (https://spinbot.com/API) and [spinnerchief] (http://developer.spinnerchief.com/API_Document.aspx). Each paraphrase tool folder contains: <strong>[corpus] </strong>and<strong> [vectors]</strong> sub-folders. For [spinnerchief], two variations are included, with 4-word-chaging ratio (default) and 2-word-chaging ratio. <strong>[vectors] sub-folder</strong> contains the average of all word vectors for each paragraph. Each line has the number of dimensions of the word embeddings technique used (see paper for more details) followed by its respective class (i.e., label mg or og). Each file belongs to one class, either "mg" or "og". The values are comma-separated (.csv). The extension is .arff can be read as a normal .txt file. The word embedding technique used is described in the file name with the following structure: &lt;technique&gt;-&lt;type&gt;-mean-&lt;data&gt;.arff . Where <em>&lt;technique&gt;</em> - d2v - doc2vec<br> google - word2vec<br> fasttextnw - fastText without subwording<br> fasttextsw - fastText with subwording<br> glove - Glove<br> <br> Details for each technique used can be found in the paper.<br> <br> <em>&lt;type&gt; - </em> arxivp - arXiv paragraph split<br> thesisp - Theses paragraph split<br> wikip - Wikipedia paragraph split (wikipedia_paragraph_vector_train are the vectors used for training. It follows the same wikip structure) Details for each technique used can be found in the paper referenced at the start of this README file. <strong>[corpus] sub-folder:</strong> contains de raw text (No pre-processing) used for train and test at a paragraph level. The Spun paragraphs used for <strong>training</strong> are only generated using the <strong>SpinBot tool</strong>. For test both SpinBot and SpinnerChief are used. The paragraph split is generated by selecting paragraphs from the original documents with 3 or more sentences. Each folder is divided in mg (i.e., machine-generated through SpinBot and SpinnerChief) and og (i.e., original-generated file). the document split is not avaiable since our experiments only use the paragraph level. Machine Learning models: SVM, Naive Bayes, and Logistic Regression. The grid search for hyperparameter adjustments for the machine learning classifiers is described in the paper. @incollection{WahleRFM22,<br> title = {Identifying {{Machine-Paraphrased Plagiarism}}},<br> booktitle = {Information for a {{Better World}}: {{Shaping}} the {{Global Future}}},<br> author = {Wahle, Jan Philip and Ruas, Terry and Folt{\’y}nek, Tom{\’a}{\v s} and Meuschke, Norman and Gipp, Bela},<br> editor = {Smits, Malte},<br> year = {2022},<br> volume = {13192},<br> pages = {393--413},<br> publisher = {{Springer International Publishing}},<br> address = {{Cham}},<br> doi = {10.1007/978-3-030-96957-8_34},<br> isbn = {978-3-030-96956-1 978-3-030-96957-8},<br> } For our previous publication using only SpinBot and Wikipedia articles for document and paragraph split, please see the following publication. The dataset used is hosted in DeepBlue <br>