Dataset and Models for Detection of News Agency Releases in Historical Newspapers

This record contains the annotated datasets and models used and produced for the work reported in the Master Thesis "Where Did the News come from? Detection of News Agency Releases in Historical Newspapers " (link). Please cite this report if you are using the models/datasets or find it relevant to your research: @article{Marxen:305129, title = {Where Did the News Come From? Detection of News Agency Releases in Historical Newspapers}, author = {Marxen, Lea}, pages = {114p}, year = {2023}, url = {http://infoscience.epfl.ch/record/305129}, } 1. DATA The newsagency-dataset contains historical newspaper articles with annotations of news agency mentions. The articles are divided into French (fr) and German (de) subsets and a train, dev and test set respectively. The data is annotated at token-level in the CoNLL format with IOB tagging format. The distribution of articles in the different sets is as follows: Dataset Statistics Lg. Docs Agency Mentions Train de 333 493 fr 903 1,122 Dev de 32 26 fr 110 114 Test de 32 58 fr 120 163 Due to an error, there are seven duplicated articles in the French test set (article IDs: courriergdl-1847-10-02-a-i0002, courriergdl-1852-02-14-a-i0002, courriergdl-1860-10-31-a-i0016, courriergdl-1864-12-15-a-i0005, lunion-1860-11-27-a-i0004, lunion-1865-02-05-a-i0012, lunion-1866-02-16-a-i0009). 2. MODELS The two agency detection and classification models used for the inference on the impresso Corpus are released as well: newsagency-model-de: based on German BERT (with maximum sequence length 128), fine-tuned with the German training set of the newsagency-dataset newsagency-model-fr: based on French Europeana BERT (with maximum sequence length 128), fine-tuned with the French training set of the newsagency-dataset The models perform multitask classification with two prediction heads, one for token-level agency entity classification and one for sentence-level (has_agency: yes/no). They can be run with TorchServe, for details see the newsagency-classification repository. Please refer to the report for further information or contact us. 3. CODE https://github.com/impresso/newsagency-classification 4. CONTACT Maud Ehrmann (EPFL-DHLAB) Emanuela Boros (EPFL-DHLAB)