Lexica corpus (v2.0)

Second release of the lexica corpus: a corpus for German text simplification, total size now 3270 files.

The corpus consists of approximately 3300 texts from three Wiki-based lexica in German language: MiniKlexikon, Klexikon and Wikipedia. The articles in the Wikis are created by volunteers and can be written, discussed, and improved upon collaboratively. Klexikon is aimed specifically at children aged between 6 and 12 and MiniKlexikon is designed for children who are beginner readers, and is therefore an even simpler version of the Klexikon. We make the assumption that the three different sub-corpora represent three different levels of conceptual complexity due to the target groups they are written for: younger children, children and adults. As Wikipedia articles can be extremely long, in comparison to the other two lexica, only the introduction or abstract was taken for this corpus.

This repository contains the corpora from the original study (295 texts per sub-corpus in the orig_files folder), extended versions with 1090 texts (as of March 2022) per sub-corpus and a script to update the extended version as new articles are added to the Klexikon and MiniKlexikon.