6000 ground truth of VOC and notarial deeds 3.000.000 HTR of VOC, WIC and notarial deeds

The National Archives of the Netherlands and Noord-Hollands Archief conducted a project using the Transkribus HTR (Handwritten Text Recognition) platform. The aim was to semi automatically transcribe 2 million pages of old Dutch texts.

The transcribed archives are 17th and 18th century documents from the Dutch East-India Company (VOC). And 19th century notarial deeds from Noord-Hollands Archief and other archives in the provinces.

In order to train the HTR software a team produced transcriptions of approximately 6000 scans. The scans are randomly selected from the dataset. With the transcriptions a model is trained that can recognize more than 90% of the characters correctly. Transkribus transcribed the 2 million scans automatically using the trained model.

The following Transkribus HTR+ model has been trained for the text recognition: "IJsberg". More information about the model can be found here. See the chapter "Dutch Handwriting". However, the Transkribus team retrained the model with PyLaia technology, which improved the HTR+ model. This PyLaia model is not publicly available.

Later on, 1 million extra scans concerning the West India Company (WIC) were transcribed automatically without adding extra ground truth or training. These archives are from the 17th and 18th century.

The Loghi Handwritten Text Recognition Toolkit has been added to the arsenal of the Nation Archives of the Netherlands. 1.05.11.14, Notarissen Suriname tot 1828 [digitaal duplicaat] has been processed with this tooling.

The datasets published in Zenodo contain the ground truth (scans in JPG, transcription in PAGE XML) and the HTR results (in PAGE XML and TXT). See the overview below. Scroll to the bottom of the page to download the actual files.

For more information on how the Dutch National Archive innovate on digital accessibility click here.

For open data access of scans and inventories of the National Archives click here.

Disclaimer: due to a variety of languages used and the bad state of the documents the HTR results of "1.05.21, Dutch series Guyana" can be of poor quality.

--------------------------------------------------------------

Dataset HTR (Dataset, name archive, number archive, inventory numbers, link to inventory)The National Archives of the NetherlandsHTR results VOC, VOC, 1.04.02, 7527-9540, EADHTR results 1.04.02, Oost-Indische Testamenten, 1.04.02, 6847-6897, EAD HTR results 1.05.01.01, Oude WIC, 1.05.01.01, 1-87,  EADHTR results 1.05.01.02, Tweede WIC, 1.05.01.02, 1-1382, EAD     HTR results 1.05.02, Raad der Koloniën, 1.05.02, 1-192, EADHTR results 1.05.03, Sociëteit van Suriname, 1.05.03, 1-566, EADHTR results 1.05.05, Sociëteit van Berbice, 1.05.05, 1-445,    EADHTR results 1.05.06, Verspreide West-Indische stukken, 1.05.06, 1-1413, EADHTR results 1.05.21, Dutch series Guyana, 1.05.21, AB.1.1-BB.7.1, EADHTR results 2.01.28.01, West-Indisch comité, 2.01.28.01, 1-254, EADHTR results 2.01.28.02, Raad der Amerikaanse Bezittingen, 2.01.28.02, 1-264, EADHTR results 1.05.11.14, Notarissen Suriname tot 1828 [digitaal duplicaat], EADHTR results 2.10.02, Koloniën, EAD (indices only)

Noord-Hollands archiefHTR results NHA Notarial 1617, Oud notarieel archief Haarlem, 1617,1593-1805, EADHTR results NHA Notarial 1972, Nieuw notarieel archief Haarlem, 1972, 5-813, EAD

Brabants Historisch Informatie CentrumHTR results BHIC 7048 , Notarissen in Boxmeer, 1814-1935, 7048, 1-103, 162, EADHTR results BHIC 7128 , Notarissen in Grave, 1648-1935, 7128, 140-266, EADHTR results BHIC 7637 , Notarissen in Sint-Oedenrode, 1642-1935, 7637, 17-78A, EAD

Gelders ArchiefHTR results GA 0168, Notariële Archieven 1811-1925, 168, 64-69, 943-960, 1366-1395, 2472-2501, 3481-3485, 3904-3926, EAD

Groninger ArchievenHTR results GRA 85, Notarissen te Appingedam (standplaats 1), 1811-1935, 85, 2-157, EADHTR results GRA 86, Notarissen te Appingedam (standplaats 2), 1812-1922, 86, 2-71, EAD

Historisch Centrum OverijsselHTR results HCO 0122, Notarissen in Overijssel, 122, 5-48, 2044-2073, 3019-3047, 3733-3775, EAD

The Utrecht ArchivesHTR results HUA 34-1, Notarissen in de provincie Utrecht, 1617-1895, 34-1, 928-930, 2209-2330, EAD

Regionaal Historisch Centrum LimburgHTR results RHCL 09.009, Notarissen in de Arrondissementen Maastricht en Roermond, 1896-1905, 09.009, 9147-9279, EAD

TresoarHTR results Tresoar 26, Notarieel archief, 26, 1001-9028 (met hiaten), EAD

Zeeuws ArchiefHTR results ZA 13.2, Notariële Archieven Zeeland 1906-1915, (1886) 1906-1915 (1925), 13.2, 1152-1163, 1261-1320, EAD

Drents ArchiefHTR results DA 114.10, Notaris jhr.mr. J.A.G.van der Wijck te Assen, 114.10, 4-7, EADHTR results DA 114.11, Notaris mr. D.A.M.de Fremery te Assen, 114.11, 1, EADHTR results DA 114.18, Notaris mr. Warmolt van Roijen te Borger, 114.18, 2-7, EADHTR results DA 114.19, Notaris mr. Ernst Sigismund. Cornets de Groot te Borger, 114.29, 2-8, EADHTR results DA 114.22, Notaris mr. Albertus Slingenberg te Coevorden, 114.22, 1-24, EADHTR results DA 114.23, Notaris mr. Gozewienus Weys te Coevorden, 114.23, 4-13, EADHTR results DA 114.28, Notaris mr. Johannes Beckeringh van Loenen te Dwingeloo, 114.29, 7-13, EADHTR results DA 114.39, Notaris mr. Gerrit ten Raa ten Gieten, 114.39, 6-14, EADHTR results DA 114.45, Notaris mr. Hendrik Jan Carsten te Hoogeveen, 114.45, 17-36, EADHTR results DA 114.54, Notaris mr. Warmold Lunsingh Tonckens te Meppel, 114.54, 18-26, EAD

Dataset Ground Truth(Name archive, number archive, inventory numbers, link to inventory, type of dataset)

Dataset: Notarial deeds Ground Truths of the trainingset

Oud notarieel archief Haarlem, 1617, 495 random scans from 1593-1805, EAD, GT Transcriptions

Nieuw notarieel archief Haarlem, 1972, 952 random scans from 5-813, EAD, GT Transcriptions

(And 168 transcripties from 7 other archives.)

Dataset: Notarial deeds Images of the trainingset,

Nieuw notarieel archief Haarlem, 1972, 952 random scans from 5-813, EAD, GT Scans

Oud notarieel archief Haarlem, 1617, 495 random scans from 1593-1805, EAD, GT Scans

(And 168 scans from 7 other archives.)

Dataset: VOC Ground Truths of the trainingset,VOC, 1.04.02, 4735 random  scans from 7527-9540, EAD, GT Transcriptions

Dataset: VOC Images of the trainingset,VOC, 1.04.02, 4735 random  scans from 7527-9540, EAD, GT Scans

--------------------------------------------------------------

Version 3.0: The first HTR results from the VOC-collection are available in .txt format, Inventory numbers 7527-9540.

Version 3.1: The HTR results from the VOC-collection are also available in PAGE xml format. 

Version 4.0: About 30 missing inventory numbers have been added to the VOC transcriptions. The HTR results of the Notarial Deeds from the NHA archives have been added. An example on full text searchable research can be found here (Dutch): https://kia.pleio.nl/groups/view/55812425/htr-en-ocr/blog/view/55814752/reconstructie-van-een-verijdelde-slavenopstand-met-behulp-van-automatische-handschriftherkenning-en-text-mining

Version 5.0: Around a million pages of HTR results of the following archives have been added.

Version 6.0: The HTR results of Oost-Indische Testamenten have been added. 

Version 7.0: The HTR results of the Brabants Historisch Informatie Centrum, Gelders Archief, Groninger Archieven, Historisch Centrum Overijssel, The Utrecht Archives, Regionaal Historisch Centrum Limburg, Tresoar, Zeeuws Archief and Drents Archief have been added.

Version 7.1: A spreadsheet "ijsberg train-val.xlsx" has been added. The division of the training- and validationset of Ground Truth of the IJsberg model can be found here

Version 8.0: HTR results of 1.05.11.14 have been added. The scans have been inferenced with Loghi.

Version 8.1: HTR results of 2.10.02 indices have been added. The scans have been inferenced with Loghi.