Dutch Historical Word2Vec models
收藏资源简介:
<strong>Introduction</strong> The repository contains Word2Vec models trained on Dutch historical newspaper data converting the period from 1840 to 1890. Models were created as part of a Research-in-Residence at the Dutch National Library. During my residency, I created language models trained on specific subsections of the newspaper corpus, to explore bias over time and by place or political leaning. To read more about this project, please read the introductory blog post. <strong>Code</strong> The code used for training the models is available on Github. Please look at the README for more instruction. <strong>Warning: </strong>the raw text data used was provided by Mirjam Cuper of the KB and is available only on request. Some code for loading and exploring the models is also available on Github. For more information on <strong>interactive lexicon creation using these models</strong>, go to this README. For more information on exploring <strong>bias on these model</strong>, go to this README. <strong>Models</strong> Models are available in zip files, one for each decade. We trained models using a window size of twenty years and a step size of five. The structure of the file names is as follows: <strong>{year_start}-{year_end}-{attribute}.w2v.model .</strong> For example <strong>1840-1860-Protestant.w2v.model</strong> is trained on all article published in protestant newspapers between 1840 and 1860. The <strong>attribute</strong> value is chosen from either the <strong>Politiek</strong> (political leaning) or <strong>Provincie</strong> (province) column in this metadata file.
<strong>简介</strong> 本仓库收录基于1840至1890年荷兰历史报纸数据训练得到的Word2Vec模型。本批模型为荷兰国家图书馆驻馆研究计划(Research-in-Residence)的产出成果。驻馆研究期间,笔者基于报纸语料库的特定子数据集训练语言模型,以探究不同时间、地域及政治倾向维度下的文本偏差情况。欲了解该项目更多细节,请参阅项目介绍博客文章。 <strong>代码</strong> 模型训练代码已上传至GitHub,更多操作指引请参阅项目README文件。 <strong>警告</strong>:本次使用的原始文本数据由荷兰皇家图书馆(KB)的米尔扬·库珀(Mirjam Cuper)提供,仅可通过申请获取。另有部分用于加载与探索模型的代码也已上传至GitHub。若需了解基于此类模型的交互式词典构建(interactive lexicon creation)更多细节,请参阅该README文件;若需了解基于此类模型的文本偏差探索(exploring bias)更多细节,同样请参阅该README文件。 <strong>模型文件</strong> 模型以ZIP压缩包形式提供,每十年对应一个压缩包。本次训练采用的窗口大小为20年,步长为5年。模型文件名格式如下:<strong>{起始年份}-{结束年份}-{属性}.w2v.model</strong>。例如,<strong>1840-1860-Protestant.w2v.model</strong> 是基于1840至1860年间新教报纸刊载的全部文章训练得到的模型。文件名中的属性(attribute)取值,可从此元数据文件中的政治倾向(Politiek)或省份(Provincie)列中选取。



