ParaPat: The Multi-Million Sentences Parallel Corpus of Patents Abstracts

This repository contains the developed parallel corpus from the open access Google Patents dataset in 74 language pairs, comprising more than 68 million sentences and 800 million tokens. Sentences were automatically aligned using the Hunalign algorithm for the largest 22 language pairs, while the others were abstract (i.e. paragraph) aligned. We demonstrate the capabilities of our corpus by training Neural Machine Translation (NMT) models for the main 9 language pairs, with a total of 18 models. Our parallel corpus is freely available in TSV format.<br>The original article can be found here: https://www.aclweb.org/anthology/2020.lrec-1.465/<br>Please cite as:<br><pre>@inproceedings{soares-etal-2020-parapat, title = "{P}ara{P}at: The Multi-Million Sentences Parallel Corpus of Patents Abstracts", author = "Soares, Felipe and Stevenson, Mark and Bartolome, Diego and Zaretskaya, Anna", booktitle = "Proceedings of The 12th Language Resources and Evaluation Conference", month = may, year = "2020", address = "Marseille, France", publisher = "European Language Resources Association", url = "https://www.aclweb.org/anthology/2020.lrec-1.465", pages = "3769--3774", abstract = "The Google Patents is one of the main important sources of patents information. A striking characteristic is that many of its abstracts are presented in more than one language, thus making it a potential source of parallel corpora. This article presents the development of a parallel corpus from the open access Google Patents dataset in 74 language pairs, comprising more than 68 million sentences and 800 million tokens. Sentences were automatically aligned using the Hunalign algorithm for the largest 22 language pairs, while the others were abstract (i.e. paragraph) aligned. We demonstrate the capabilities of our corpus by training Neural Machine Translation (NMT) models for the main 9 language pairs, with a total of 18 models. Our parallel corpus is freely available in TSV format and with a SQLite database, with complementary information regarding patent metadata.", language = "English", ISBN = "979-10-95546-34-4", }</pre>