JuVer Contract Corpus

This corpus consists of 2110 PDF Files and 2110 XML files with the text extracted from the PDF files. All PDF files are contracts in German publically available on the internet. Most of these contracts are from the city governments of Hamburg and Bremen and were collected from the websites http://suche.transparenz.hamburg.de/dataset?q=vertrag&amp;esq_title=&amp;check_all_ and https://www.transparenz.bremen.de. In the XML files the texts are segmented into sentences. Each sentence also has some additional information on the freuency of use in the corpus. The root of each XML file is the element <em>document</em>, that has a referece to the original PDF in an attribute. A document is divided into pages. Pages then consists of the elements <em>heading </em>and <em>sentence</em>. Each sentence has two identifiers, sid for the sentece and cid for the cluster it belongs to. Sentenecs with the same sentence identifier are identical. Sentences with the same cluster identifier are very similar but not necessarily identical. Sentences were clustered with single link clustering based on trigram (character) overlap. The corpus consists of 106,539 (non-unique) sentences and 3,635,371 tokens, including interpunction.