runetrust/blame-folketinget-dk

---
language:
- da
pretty_name: Folketinget Denmark with Blame Labels
size_categories:
- 1M<n<10M
license: cc0-1.0
tags:
- politics
- debates
dataset_info:
features:
- name: text
dtype: string
- name: labels
dtype: int64
- name: speaker
dtype: string
- name: date
dtype: string
- name: party
dtype: string
- name: paragraph_nr
dtype: int64
- name: sentence_nr
dtype: int64
splits:
- name: train
num_bytes: 96009232
num_examples: 450000
- name: test
num_bytes: 93399
num_examples: 424
- name: validation
num_bytes: 10653303
num_examples: 50000
- name: inference
num_bytes: 1085835611
num_examples: 5098494
download_size: 464059031
dataset_size: 1192591545
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
- split: test
path: data/test-*
- split: validation
path: data/validation-*
- split: inference
path: data/inference-*
---
# Label Scheme:
## Blame: 1
## No Blame: 0

This dataset was created partially from The ParlSpeech V2 data set (citation below) from 07/10-1997 up to and including 20/12-2018, with data from 08/01-2019 up to and including 26/02-2026 coming from our own fetch from Folketingets own sFTP server.

Blame labels were created on a machine translated version of the data (using opus-MT) by the Political DEBATE model found here: https://huggingface.co/mlburnham/Political_DEBATE_large_v1.0

The train / validation split is a simple 90/10 split of the machine labelled data, with the test set being manually annotated by two authors with an inter-annotator agreement of 84.8% - a Cohen's Kappa of 0.675.

Inference data is what remained after cleaning, as described in the paper {forthcoming}, and extraction of training data.

More details can be found in the GitHub for the project here: {Anonymized}

Rauh, C., & Schwalbach, J. (2020). The ParlSpeech V2 data set: Full-text corpora of 6.3 million parliamentary speeches in the key legislative chambers of nine representative democracies (Version V1) [Data set]. Harvard Dataverse. https://doi.org/10.7910/DVN/L4OAKN