Dutch Historical Word2Vec models

Introduction

The repository contains Word2Vec models trained on Dutch historical newspaper data converting the period from 1840 to 1890. Models were created as part of a Research-in-Residence at the Dutch National Library. During my residency, I created language models trained on specific subsections of the newspaper corpus, to explore bias over time and by place or political leaning.

To read more about this project, please read the introductory blog post.

Code

The code used for training the models is available on Github. Please look at the README for more instruction. 

Warning: the raw text data used was provided by Mirjam Cuper of the KB and is available only on request.

Some code for loading and exploring the models is also available on Github.

For more information on interactive lexicon creation using these models, go to this README.

For more information on exploring bias on these model, go to this README.

Models

Models are available in zip files, one for each decade. We trained models using a window size of twenty years and a step size of five. The structure of the file names is as follows:

{year_start}-{year_end}-{attribute}.w2v.model .

For example 1840-1860-Protestant.w2v.model is trained on all article published in protestant newspapers between 1840 and 1860.

The attribute value is chosen from either the Politiek (political leaning) or Provincie (province) column in this metadata file.