GermaParl Corpus of Plenary Protocols

The GermaParl Corpus of Parliamentary Protocols covers 74 years of debates in the German Bundestags, from the first meeting on September 7, 1949 up to the last meeting of the 19th legislative period on July 7, 2023. The corpus has been prepared in the PolMine Project, building on an initial GermaParl version covering the period between 1996 and 2016. The public release of GermaParl v2.1.0 is available here under under a Creative Commons license (CC BY-SA 4.0). To learn more about the data, see the documentation of the corpus available here. The essence of GermaParl is a transformation of the raw material available from the Bundestag (plaint text, PDF and XML documents) into an XML data format that approximates standards envisaged by the Text Encoding Initiative (TEI). Aside from these XML documents here (and on GitHub), we provide an indexed and linguistically annotated CWB version of the corpus. Based on XML files, parliamentary proceedings have been linguistically annotated (using Stanford CoreNLP and further tools) and been converted into the binary data format of the Corpus Workbench (CWB). The corpus is intended to be a trustworthy, high-quality resource for research on parliamentary proceedings. While GermaParl v2.1.0 is a considerably large dataset (4461 plenary protocols / 281 million words), using the CWB version in combination with polmineR provides fast and free data access for everybody. All that is required is to install R and the packages polmineR and cwbtools, and the corpus itself. Download and install corpus: We suggest to install the corpus using functionality included in the R package cwbtools. Make sure that you have cwbtools v0.3.8 or higher. Use the code snippet provided below. A proper internet connection is advisable: The size of the corpus tarball is 2,5 GB. install.packages(pkgs = c("cwbtools", "polmineR")) cwbtools::corpus_install(doi = "10.5281/zenodo.12794676") library(polmineR) corpus("GERMAPARL2") If you have not used CWB indexed corpora before, the installation process will suggest and create directories for data storage. This involves defining the environment variable CORPUS_REGISTRY permanently for future R sessions. Explore GermaParl and give feedback: We have taken great care to offer a high-quality research resource. But given the size of the data, it is impossible to manually check the data throughout. We encourage every user of GermaParl2 to contribute to the quality of the resource by reporting bugs and flaws in the corpus. To let us know how the data can be improved, we use GitHub issues as our primary issue tracker. Issues concerning GermaParl v2 are collected in the GermaParl2 GitHub repository (https://github.com/PolMine/GermaParl2). You are also welcome to send us your suggestions via e-mail ([e-mail]). Acknowledgements: We gratefully acknowledge funding from the German National Research Data Infrastructure (Nationale Forschungsdaten-Infrastruktur / NFDI). Funding from KonsortSWD (project number 442494171) has advanced the data preparation tool set to facilitate the robust annotation of additional annotation layers in large corpora (such as Named Entities). This is instrumental for linking parliamentary data with other data. KonsortSWD is funded by the German Research Foundation (DFG) as part of the National Research Data Infrastructure Germany (Nationale Forschungsdateninfrastruktur, NFDI) under project number 442494171. Funding from the Text+ consortium is instrumental for updates of the corpus, quality control and keeping data formats up with current and future developments. Text+ is funded by the German Research Foundation (DFG) as part of the NFDI under project number 460033370. The data quality of GermaParl we are able to offer at this stage has benefitted significantly from a cooperation with the SOLDISK project at the University of Hildesheim, and comprehensive manual quality control of the data carried out by the SOLDISK team. A very special thanks goes to Hannes Schammann, Max Kisselew, Franziska Ziegler, Carina Böker, Jennifer Elsner and Carolin McCrea. We also would like to thank our beta users which provided us with invaluable feedback and greatly enhanced the quality of the data over the course of multiple release candidates.《GermaParl议会议事记录语料库》收录了1949年9月7日德国联邦议院首次会议至2023年7月7日第19届立法周期最后一次会议期间，共计74年的议事辩论内容。本语料库由PolMine项目开发制作，其前身是覆盖1996年至2016年时段的初代GermaParl版本。 GermaParl v2.1.0的公开版本可在此处获取，采用知识共享署名-相同方式共享4.0（Creative Commons Attribution-ShareAlike 4.0，CC BY-SA 4.0）许可协议。如需了解该数据集的更多信息，请查阅此处提供的语料库文档。 GermaParl的核心工作是将德国联邦议院公开的原始素材（纯文本、PDF及XML文档）转换为符合文本编码倡议（Text Encoding Initiative，TEI）拟定标准的XML数据格式。除了此处（及GitHub平台）提供的XML文档外，本项目还提供该语料库的索引化语言标注版语料库工作台（Corpus Workbench，CWB）版本。该语料库以XML文件为基础，采用Stanford CoreNLP及其他工具完成语言标注，并转换为CWB的二进制数据格式。 本语料库旨在为议会议事研究提供可靠、高质量的研究资源。尽管GermaParl v2.1.0是体量庞大的数据集（包含4461份全体会议记录，共计2.81亿词元），但通过将CWB版本与polmineR包结合使用，所有用户均可快速、免费地获取数据。仅需安装R语言环境、polmineR与cwbtools两个R包，以及本语料库即可使用。 语料库下载与安装：我们建议通过R包cwbtools内置的功能完成语料库安装，请确保使用的cwbtools版本不低于v0.3.8。可使用下文提供的代码片段进行安装。建议使用稳定的网络连接：本语料库压缩包的大小为2.5GB。 r install.packages(pkgs = c("cwbtools", "polmineR")) cwbtools::corpus_install(doi = "10.5281/zenodo.12794676") library(polmineR) corpus("GERMAPARL2") 若您此前未使用过CWB索引化语料库，安装过程将自动提示并创建数据存储目录，同时会为后续R会话永久配置`CORPUS_REGISTRY`环境变量。 使用GermaParl并反馈问题：我们始终致力于提供高质量的研究资源，但受限于数据集的庞大规模，无法对全部数据进行人工核验。我们鼓励所有GermaParl2用户通过上报语料库中的错误与缺陷，共同提升本资源的质量。我们以GitHub Issues作为主要的问题追踪渠道，与GermaParl v2相关的问题可提交至GermaParl2的GitHub仓库（https://github.com/PolMine/GermaParl2）。您也可通过电子邮件（[e-mail]）向我们发送改进建议。 致谢： 我们衷心感谢德国国家研究数据基础设施（Nationale Forschungsdaten-Infrastruktur，NFDI）提供的资助。KonsortSWD（项目编号442494171）提供的资金支持升级了数据准备工具集，使其能够为大型语料库的额外标注层（如命名实体（Named Entities））提供可靠的标注服务，这对议会数据与其他数据的关联工作至关重要。KonsortSWD由德国研究基金会（Deutsche Forschungsgemeinschaft，DFG）作为德国国家研究数据基础设施（NFDI）的一部分予以资助，项目编号为442494171。 Text+联盟提供的资金支持对语料库的更新、质量控制以及保持数据格式适配当前及未来的发展需求至关重要。Text+由德国研究基金会（DFG）作为NFDI的一部分予以资助，项目编号为460033370。 现阶段我们所能提供的GermaParl数据质量，得益于与希尔德斯海姆大学SOLDISK项目的合作，以及SOLDISK团队对数据开展的全面人工质量核验。在此特别感谢Hannes Schammann、Max Kisselew、Franziska Ziegler、Carina Böker、Jennifer Elsner及Carolin McCrea。 我们还要感谢所有公测用户，他们提供了宝贵的反馈意见，在多个预发布版本的迭代过程中极大地提升了本数据集的质量。