Models for "A data-driven approach to studying changing vocabularies in historical newspaper collections"
收藏资源简介：
NOTE: This is a badly rendered version of the README within the archive. <strong>A data-driven approach to studying changing vocabularies in historical newspaper collections</strong> Simon Hengchen,* Ruben Ros,** Jani Marjanen,*** Mikko Tolonen*** *Språkbanken Text, University of Gothenburg, Sweden and iguanodon.ai, Belgium: firstname.lastname@gu.se<br> **Centre for Contemporary and Digital History (C2DH), University of Luxembourg: firstname.lastname@uni.lu<br> ***COMHIS, University of Helsinki: firstname.lastname@helsinki.fi; These are the supplementary materials for the DH2019 paper <em>A data-driven approach to the changing vocabulary of the ‘nation’ in English, Dutch, Swedish and Finnish newspapers, 1750-1950</em>, as well as the 2021 Digital Scholarship in the Humanities publication available in OpenAccess: https://academic.oup.com/dsh/article/36/Supplement_2/ii109/6421793. If you end up using whole or parts of this resource, please use the following citation(s): Hengchen, S., Ros, R., and Marjanen, J. (2019). A data-driven approach to the changing vocabulary of the 'nation' in English, Dutch, Swedish and Finnish newspapers, 1750-1950. In <em>Proceedings of the Digital Humanities (DH) conference 2019, Utrecht, The Netherlands</em> and/or: Hengchen, S., Ros, R., Marjanen, J. and Tolonen, M., 2021. A data-driven approach to studying changing vocabularies in historical newspaper collections. Digital Scholarship in the Humanities, 36(Supplement_2), pp.ii109-ii126. or alternatively use one of the following <code>bib</code>s: <pre><code>@inproceedings{hengchen2019nation, title="A data-driven approach to the changing vocabulary of the 'nation' in {E}nglish, {D}utch, {S}wedish and {F}innish newspapers, 1750-1950.", author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani}, year={2019}, address = "Utrecht, The Netherlands", booktitle={Proceedings of the Digital Humanities (DH) conference 2019} }</code></pre> <pre><code>@article{hengchen2021data, title={A data-driven approach to studying changing vocabularies in historical newspaper collections}, author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani and Tolonen, Mikko}, journal={Digital Scholarship in the Humanities}, volume={36}, number={Supplement\_2}, pages={ii109--ii126}, year={2021}, publisher={Oxford University Press} }</code></pre> Files This archive contains two folders -- one per diachronic representation method -- as well as this README. The folders each contain four folders, which contain the models for their respective languages. As can be inferred from the small datasize, most of the earlier models are not reliable and should not be used, but are still made available. This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License. <strong>Source material</strong> Finnish: The models were created with data from the Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland (National Library of Finland, 2011). We used everything in the corpus. Filesizes: <pre><code>[simon@taito-login3 SGNS]$ du -h fi* 12M fi_1820_SGNS_corpus_file.gensim 89M fi_1840_SGNS_corpus_file.gensim 797M fi_1860_SGNS_corpus_file.gensim 7.0G fi_1880_SGNS_corpus_file.gensim 22G fi_1900_SGNS_corpus_file.gensim</code></pre> Swedish: The models were created with data from the Kubhist 2 corpus (Språkbanken) -- more precisely, the data dumps available at https://spraakbanken.gu.se. After a manual evaluation of Swedish embeddings trained without pre-processing seemed to show that the embeddings were of low quality, we retrained models, only keeping sentences that were at least 10 tokens long and were constituted of at least 50% of lemmas as per the KORP processing pipeline (Borin et al, 2012). Filesizes: <pre><code>[simon@taito-login3 SGNS]$ du -h sv* 1.6M sv_1740_SGNS_corpus_file.gensim 44M sv_1760_SGNS_corpus_file.gensim 124M sv_1780_SGNS_corpus_file.gensim 228M sv_1800_SGNS_corpus_file.gensim 678M sv_1820_SGNS_corpus_file.gensim 1.6G sv_1840_SGNS_corpus_file.gensim 4.5G sv_1860_SGNS_corpus_file.gensim 6.5G sv_1880_SGNS_corpus_file.gensim 113M sv_1900_SGNS_corpus_file.gensim</code></pre> Dutch: The models were created with data from the Delpher newspaper archive (Royal Dutch Library, 2017), through data dumps for newspapers until and including 1876, and through API hits for articles from 1877 to 1899 (included). For anything pre-1877 we discarded full texts that had, in the metadata, anything else than exclusively <code>nl</code> or <code>NL</code> as a language tag. For the full texts between 1877 and 1899: we queried the API for all items in the “artikel” category that contained the determiner <code>de</code>. Our assumption was that most articles should contain <code>de</code> at least once, and those that didn't were too short to be deemed interesting. A subsequent study showed that was not exactly the case, but we were reassured by the fact that left-out articles were probably "shipping or financial reports" (thanks go to Melvin Wevers). We also did not include the colonial newspapers for our embeddings. This is motivated by our research questions. A list of removed newspapers is available on request. Filesizes: <pre><code>[simon@taito-login3 SGNS]$ du -h nl* 6.8M nl_1620_SGNS_corpus_file.gensim 7.9M nl_1640_SGNS_corpus_file.gensim 43M nl_1660_SGNS_corpus_file.gensim 78M nl_1680_SGNS_corpus_file.gensim 138M nl_1700_SGNS_corpus_file.gensim 243M nl_1720_SGNS_corpus_file.gensim 287M nl_1740_SGNS_corpus_file.gensim 431M nl_1760_SGNS_corpus_file.gensim 825M nl_1780_SGNS_corpus_file.gensim 1.2G nl_1800_SGNS_corpus_file.gensim 1.8G nl_1820_SGNS_corpus_file.gensim 3.1G nl_1840_SGNS_corpus_file.gensim 5.2G nl_1860_SGNS_corpus_file.gensim 13G nl_1880_SGNS_corpus_file.gensim</code></pre> English: The models were created with data from the British Library Newspapers collection (link), the Nichols collection (link), and the Burney collection (link). We used everything in the corpora. For English, only SGNS_ALIGN models are available. We thank Gale Cengage for their help with this project. Filesizes: <pre><code>[simon@taito-login3 SGNS]$ du -h en* 4.3M en_1620_SGNS_corpus_file.gensim 11M en_1640_SGNS_corpus_file.gensim 11M en_1660_SGNS_corpus_file.gensim 106M en_1680_SGNS_corpus_file.gensim 409M en_1700_SGNS_corpus_file.gensim 1.7G en_1720_SGNS_corpus_file.gensim 834M en_1740_SGNS_corpus_file.gensim 2.4G en_1760_SGNS_corpus_file.gensim 5.3G en_1780_SGNS_corpus_file.gensim 5.5G en_1800_SGNS_corpus_file.gensim 15G en_1820_SGNS_corpus_file.gensim 42G en_1840_SGNS_corpus_file.gensim 65G en_1860_SGNS_corpus_file.gensim 88G en_1880_SGNS_corpus_file.gensim 26G en_1900_SGNS_corpus_file.gensim 21G en_1920_SGNS_corpus_file.gensim 6.3G en_1940_SGNS_corpus_file.gensim</code></pre> <strong>Word embeddings</strong> For every language, we train diachronic embeddings as follows. We divide the data in 20-year time bins. We train SGNS_UPDATE and SGNS_ALIGN models. Current research on German (Schlechtweg et al, 2019) and English (Shoemark et al, 2019) indicates you should use the SGNS_ALIGN models. <strong>For EN, FI, NL, no tokens (including punctuation) were removed nor altered, aside from lowercasing</strong>. For SV, see above. Parameters are as follows: SGNS architecture (Mikolov et al 2013), window size of 5, frequency threshold of 100, 5 epochs, 300 dimensions (or 100 for EN). For SGNS_UPDATE: We first train a model for the first time bin <code>t</code>. To train the model for <code>t+1</code>, we use the <code>t</code> model to initialise the vectors for <code>t+1</code>, set the learning rate to correspond to the end learning rate of <code>t</code>, and continue training. This approach, closely following Kim et al (2014), has the advantage of avoiding the need for post-training vector space alignment. The Python snippet below, which makes use of gensim (Rehurek and Sojka, 2010), illustrates the approach. Special thanks go to Sara Budts. <pre><code>## dict_files[key] is a dictionary with double decades as keys and a corresponding LineSentence object as value: https://radimrehurek.com/gensim/models/word2vec.html#gensim.models.word2vec.LineSentence count = 0 for key in sorted(list(dict_files.keys())): if count == 0: ## This is the first model. model = gensim.models.Word2Vec(corpus_file=dict_files[key], min_count=100, sg=1 ,size=300, workers=64, seed=1830, iter=5) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) print("Model saved, on to the next\n") count += 1 if count > 0: ## this is for the subsequent models. print("model for double decade starting in",str(key)) model = gensim.models.Word2Vec.load(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin-20)+".w2v")) print("previous model loaded") model.build_vocab(corpus_file=dict_files[key], update=True) model.train(corpus_file=dict_files[key], total_words = model.corpus_count, total_examples = model.corpus_count, start_alpha = model.alpha, end_alpha = model.min_alpha, epochs=model.epochs) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) </code></pre> For SGNS_ALIGN: We independently train models for all time bins. The models in this repository are <em>NOT</em> aligned, leaving you the choice of how to align them. For example, here is a link to code by Ryan Heuser to do just that. Models were trained with the <code>count == 0</code> scenario in the snippet above. <strong>Acknowledgments</strong> This work has been supported by the European Union's Horizon 2020 research and innovation programme under grant 770299 NewsEye. Specials thanks go to the data providers/collection-holding institutions: the Finnish Language Bank, the Swedish Language Bank, the Royal Dutch Library, and Gale Cengage. The authors would like to thank the following persons and group, listed alphabetically: Antoine Doucet, Antti Kanner, Axel-Jean Caurant, Dominik Schlechtweg, Eetu Mäkelä, Elaine Zosa, Estelle Bunout, Haim Dubossarsky, Joris van Eijnatten, Krister Lindén, Lars Borin, Lidia Pivovarova, Melvin Wevers, Nina Tahmasebi, Sara Budts, Senka Drobac, Tanja Säily, the COMHIS group, and Steven Claeyssens. Computational resources were provided by CSC – IT Center for Science Ltd. <strong>References</strong> Borin, L., Forsberg, M., Roxendal, J. (2012). Korp-the corpus infrastructure of Spräkbanken,in: LREC. pp. 474–478. Kim, Y., Chiu, Y.I., Hanaki, K., Hegde, D. and Petrov, S. (2014). Temporal Analysis of Language through Neural Language Models. <em>ACL 2014</em>, p.61. Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). Efficient estimation of word representations in vector space. <em>arXiv preprint arXiv:1301.3781</em>. National Library of Finland (2011). <em>The Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland, Kielipankki Version</em> [text corpus]. Kielipankki. Retrieved from http://urn.fi/urn:nbn:fi:lb-2016050302. Rehurek, R. and Sojka, P. (2010). Software framework for topic modelling with large corpora. In <em>Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks</em>. Royal Dutch Library (2017). <em>Delpher open krantenarchief (1.0)</em>. Den Haag, 2017. Schlechtweg D., Hätty A, del Tredici M., and Schulte im Walde S. (2019). A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domains. In <em>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, Florence, Italy. ACL. Shoemark, P., Liza, F.F., Nguyen, D., Hale, S. and McGillivray, B. (2019). Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings. In <em>Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 66-76)</em>, Hong Kong. Språkbanken. <em>The Kubhist Corpus</em>. Department of Swedish, University of Gothenburg. https://spraakbanken.gu.se/korp/?mode=kubhist.
注意：本文件为存档内README文档的渲染异常版本。**面向历史报纸馆藏词汇演变研究的数据驱动方法** 西蒙·亨琴*，鲁本·罗斯**，亚尼·马尔亚宁***，米科·托洛宁*** * 瑞典哥德堡大学Språkbanken Text实验室及比利时iguanodon.ai：firstname.lastname@gu.se ** 卢森堡大学当代与数字历史研究中心（C2DH）：firstname.lastname@uni.lu *** 赫尔辛基大学COMHIS研究中心：firstname.lastname@helsinki.fi 本存档为2019年数字人文（DH）会议论文《1750-1950年英、荷、瑞、芬报纸中“民族”语义演变的数驱动方法》以及2021年发表于《数字人文（Digital Scholarship in the Humanities）》（开放获取链接：https://academic.oup.com/dsh/article/36/Supplement_2/ii109/6421793）的补充材料。 若您整体或部分使用本资源，请引用如下文献： 西蒙·亨琴，鲁本·罗斯，亚尼·马尔亚宁（2019）。1750-1950年英、荷、瑞、芬报纸中“民族”语义演变的数驱动方法。载于《2019年数字人文（DH）会议论文集》，荷兰乌得勒支。 西蒙·亨琴，鲁本·罗斯，亚尼·马尔亚宁，米科·托洛宁（2021）。历史报纸馆藏词汇演变研究的数据驱动方法。《数字人文（Digital Scholarship in the Humanities）》，36（增刊2），第ii109-ii126页。 亦可使用如下<code>bib</code>格式引用： <pre><code>@inproceedings{hengchen2019nation, title="A data-driven approach to the changing vocabulary of the 'nation' in {E}nglish, {D}utch, {S}wedish and {F}innish newspapers, 1750-1950.", author={Hengchen, Simon and Ros, Ruben and Marjanen, Jani}, year={2019}, address = "Utrecht, The Netherlands", booktitle={Proceedings of the Digital Humanities (DH) conference 2019} }</code></pre> <pre><code>@article{hengchen2021data, title={A data-driven approach to studying changing vocabularies in historical newspaper collections}, author={Hengchen, Simon and Ros, Ruben and Marjanen, J. and Tolonen, Mikko}, journal={Digital Scholarship in the Humanities}, volume={36}, number={Supplement_2}, pages={ii109--ii126}, year={2021}, publisher={Oxford University Press} }</code></pre> ## 文件说明 本存档包含两个文件夹（分别对应一种历时表征方法）以及本README文档。每个文件夹下均设有四个子文件夹，存储对应语言的模型。鉴于文件体积普遍偏小，多数早期模型可靠性不足，不建议使用，但仍予以公开。 本作品采用知识共享署名-相同方式共享4.0国际许可协议（Creative Commons Attribution-ShareAlike 4.0 International License）进行许可。 ### 数据源 **芬兰语**：本模型的数据源自芬兰国家图书馆报纸与期刊语料库芬兰子语料库（芬兰国家图书馆，2011），我们使用了该语料库的全部数据。 文件大小： <pre><code>[simon@taito-login3 SGNS]$ du -h fi* 12M fi_1820_SGNS_corpus_file.gensim 89M fi_1840_SGNS_corpus_file.gensim 797M fi_1860_SGNS_corpus_file.gensim 7.0G fi_1880_SGNS_corpus_file.gensim 22G fi_1900_SGNS_corpus_file.gensim</code></pre> **瑞典语**：本模型的数据源自Kubhist 2语料库（Språkbanken）——具体为https://spraakbanken.gu.se 提供的数据导出文件。经人工评估发现，未经过预处理训练的瑞典语词嵌入质量较低，因此我们重新训练了模型：仅保留长度至少为10个词元（Token）、且按照KORP处理流水线（Borin等，2012）统计的词形还原词占比不低于50%的句子。 文件大小： <pre><code>[simon@taito-login3 SGNS]$ du -h sv* 1.6M sv_1740_SGNS_corpus_file.gensim 44M sv_1760_SGNS_corpus_file.gensim 124M sv_1780_SGNS_corpus_file.gensim 228M sv_1800_SGNS_corpus_file.gensim 678M sv_1820_SGNS_corpus_file.gensim 1.6G sv_1840_SGNS_corpus_file.gensim 4.5G sv_1860_SGNS_corpus_file.gensim 6.5G sv_1880_SGNS_corpus_file.gensim 113M sv_1900_SGNS_corpus_file.gensim</code></pre> **荷兰语**：本模型的数据源自Delpher报纸档案（荷兰皇家图书馆，2017）：1876年及之前的报纸使用数据导出文件，1877年至1899年的报纸则通过API接口获取文章数据（包含该时间段）。针对1877年之前的数据，我们舍弃了元数据中语言标签非<code>nl</code>或<code>NL</code>的完整文本。针对1877年至1899年的完整文本，我们通过API接口查询了“artikel”（文章）分类下所有包含限定词<code>de</code>的条目。我们的初始假设是多数文章至少会出现一次<code>de</code>，未包含该词的文章则过短，不具备研究价值。后续研究表明该假设并不完全成立，但所幸被舍弃的文章多为“航运或财务报告”（感谢梅尔文·韦弗斯告知此细节）。此外，鉴于本研究的问题导向，我们未将殖民时期报纸纳入词嵌入训练。移除的报纸清单可应要求提供。 文件大小： <pre><code>[simon@taito-login3 SGNS]$ du -h nl* 6.8M nl_1620_SGNS_corpus_file.gensim 7.9M nl_1640_SGNS_corpus_file.gensim 43M nl_1660_SGNS_corpus_file.gensim 78M nl_1680_SGNS_corpus_file.gensim 138M nl_1700_SGNS_corpus_file.gensim 243M nl_1720_SGNS_corpus_file.gensim 287M nl_1740_SGNS_corpus_file.gensim 431M nl_1760_SGNS_corpus_file.gensim 825M nl_1780_SGNS_corpus_file.gensim 1.2G nl_1800_SGNS_corpus_file.gensim 1.8G nl_1820_SGNS_corpus_file.gensim 3.1G nl_1840_SGNS_corpus_file.gensim 5.2G nl_1860_SGNS_corpus_file.gensim 13G nl_1880_SGNS_corpus_file.gensim</code></pre> **英语**：本模型的数据源自大英图书馆报纸馆藏（链接）、尼科尔斯馆藏（链接）以及伯尼馆藏（链接），我们使用了这些语料库的全部数据。针对英语语料，本存档仅提供SGNS_ALIGN模型。我们感谢Gale Cengage为本项目提供的协助。 文件大小： <pre><code>[simon@taito-login3 SGNS]$ du -h en* 4.3M en_1620_SGNS_corpus_file.gensim 11M en_1640_SGNS_corpus_file.gensim 11M en_1660_SGNS_corpus_file.gensim 106M en_1680_SGNS_corpus_file.gensim 409M en_1700_SGNS_corpus_file.gensim 1.7G en_1720_SGNS_corpus_file.gensim 834M en_1740_SGNS_corpus_file.gensim 5.3G en_1760_SGNS_corpus_file.gensim 5.5G en_1780_SGNS_corpus_file.gensim 15G en_1800_SGNS_corpus_file.gensim 42G en_1820_SGNS_corpus_file.gensim 65G en_1840_SGNS_corpus_file.gensim 88G en_1860_SGNS_corpus_file.gensim 26G en_1880_SGNS_corpus_file.gensim 21G en_1900_SGNS_corpus_file.gensim 6.3G en_1920_SGNS_corpus_file.gensim 2.4G en_1940_SGNS_corpus_file.gensim</code></pre> ### 词嵌入（Word embeddings） 针对每种语言，我们按如下方式训练历时词嵌入：将数据按20年为一个时间窗口进行划分，分别训练SGNS_UPDATE与SGNS_ALIGN两种模型。现有针对德语（Schlechtweg等，2019）与英语（Shoemark等，2019）的研究表明，应优先选用SGNS_ALIGN模型。 **针对英语（EN）、芬兰语（FI）、荷兰语（NL），除转换为小写外，未对任何词元（Token）（包括标点符号）进行移除或修改**。针对瑞典语（SV）的预处理方式详见前文。 模型参数设置如下：采用SGNS架构（Mikolov等，2013），窗口大小为5，词频阈值为100，训练轮次为5，词嵌入维度为300（英语语料为100）。 #### SGNS_UPDATE模型 我们首先为第一个时间窗口<code>t</code>训练模型。训练后续时间窗口<code>t+1</code>的模型时，我们使用<code>t</code>时刻训练得到的模型初始化<code>t+1</code>时刻的词向量，将学习率设置为与<code>t</code>时刻训练的最终学习率一致，再继续训练。该方法严格遵循Kim等（2014）的方案，其优势在于无需在训练后进行向量空间对齐。以下Python代码片段借助gensim工具包（Rehurek与Sojka，2010）演示了该实现流程，特别感谢萨拉·布茨为此提供的帮助。 <pre><code>## dict_files[key] is a dictionary with double decades as keys and a corresponding LineSentence object as value: https://radimrehurek.com/gensim/models/word2vec.html#gensim.models.word2vec.LineSentence count = 0 for key in sorted(list(dict_files.keys())): if count == 0: ## This is the first model. model = gensim.models.Word2Vec(corpus_file=dict_files[key], min_count=100, sg=1 ,size=300, workers=64, seed=1830, iter=5) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) print("Model saved, on to the next ") count += 1 if count > 0: ## this is for the subsequent models. print("model for double decade starting in",str(key)) model = gensim.models.Word2Vec.load(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin-20)+".w2v")) print("previous model loaded") model.build_vocab(corpus_file=dict_files[key], update=True) model.train(corpus_file=dict_files[key], total_words = model.corpus_count, total_examples = model.corpus_count, start_alpha = model.alpha, end_alpha = model.min_alpha, epochs=model.epochs) model.save(os.path.join(data_path_final,"KIM",lang+"_"+str(timebin)+".w2v")) </code></pre> #### SGNS_ALIGN模型 我们为所有时间窗口独立训练模型。本存档中的模型未进行空间对齐，您可自行选择对齐方法。例如，此处提供了Ryan Heuser编写的对齐代码链接。本存档中的模型均采用前述代码片段中<code>count == 0</code>的场景进行训练。 ### 致谢 本研究得到欧盟地平线2020研究与创新计划资助（项目编号770299，NewsEye项目）。特别感谢各数据源/馆藏机构提供的支持：芬兰语言银行、瑞典语言银行、荷兰皇家图书馆以及Gale Cengage。作者谨向以下个人与研究团队致谢（按字母顺序排列）：Antoine Doucet、Antti Kanner、Axel-Jean Caurant、Dominik Schlechtweg、Eetu Mäkelä、Elaine Zosa、Estelle Bunout、Haim Dubossarsky、Joris van Eijnatten、Krister Lindén、Lars Borin、Lidia Pivovarova、Melvin Wevers、Nina Tahmasebi、Sara Budts、Senka Drobac、Tanja Säily、COMHIS研究团队以及Steven Claeyssens。计算资源由CSC——芬兰国家计算机中心有限公司提供。 ### 参考文献 1. Borin, L., Forsberg, M., Roxendal, J. (2012). Korp-the corpus infrastructure of Spräkbanken,in: LREC. pp. 474–478. 2. Kim, Y., Chiu, Y.I., Hanaki, K., Hegde, D. and Petrov, S. (2014). Temporal Analysis of Language through Neural Language Models. <em>ACL 2014</em>, p.61. 3. Mikolov, T., Chen, K., Corrado, G. and Dean, J. (2013). Efficient estimation of word representations in vector space. <em>arXiv preprint arXiv:1301.3781</em>. 4. National Library of Finland (2011). <em>The Finnish Sub-corpus of the Newspaper and Periodical Corpus of the National Library of Finland, Kielipankki Version</em> [text corpus]. Kielipankki. Retrieved from http://urn.fi/urn:nbn:fi:lb-2016050302. 5. Rehurek, R. and Sojka, P. (2010). Software framework for topic modelling with large corpora. In <em>Proceedings of the LREC 2010 Workshop on New Challenges for NLP Frameworks</em>. 6. Royal Dutch Library (2017). <em>Delpher open krantenarchief (1.0)</em>. Den Haag, 2017. 7. Schlechtweg D., Hätty A, del Tredici M., and Schulte im Walde S. (2019). A Wind of Change: Detecting and Evaluating Lexical Semantic Change across Times and Domains. In <em>Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)</em>, Florence, Italy. ACL. 8. Shoemark, P., Liza, F.F., Nguyen, D., Hale, S. and McGillivray, B. (2019). Room to Glo: A Systematic Comparison of Semantic Change Detection Approaches with Word Embeddings. In <em>Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing (EMNLP-IJCNLP) (pp. 66-76)</em>, Hong Kong. 9. Språkbanken. <em>The Kubhist Corpus</em>. Department of Swedish, University of Gothenburg. https://spraakbanken.gu.se/korp/?mode=kubhist.
