TurkuNLP/Turku-WebQA
收藏数据集概述
Turku WebQA数据集是一个芬兰语的问答数据集，从不同的CommonCrawl来源（Parsebank、mC4-Fi、CC-Fi）中提取。该数据集包含237,000个问答对（总共290,000个问题，但并非所有问题都有答案）。没有答案的问题可以通过排除None（空值）的行来丢弃。
数据字段
source：一个string特征，指示问答对是从Parsebank、mC4-Fi还是CC-Fi提取的。id：一个string特征，原始文本的ID，从中提取问答对。question：一个string特征。answer：一个string特征，也可能是None（空值）。
问答对的手动评估
为了了解提取的问答对的质量，对样本进行了噪声伪影、不足答案和缺失上下文的标注。评估显示，不同来源的语料库之间存在差异。
| 来源 | 噪声伪影 | 不足答案 | 缺失上下文 |
|---|---|---|---|
| 总计 (N=73) | 0.29 | 0.22 | 0.08 |
| CC-Fi (N=25) | 0.36 | 0.22 | 0.03 |
| mC4-Fi (N=25) | 0.28 | 0.28 | 0.14 |
| Parsebank (N=22) | 0.23 | 0.14 | 0.07 |
引用
要引用此数据集，请使用以下bibtex：
bibtex @inproceedings{eskelinen-etal-2024-building-question, title = "Building Question-Answer Data Using Web Register Identification", author = "Eskelinen, Anni and Myntti, Amanda and Henriksson, Erik and Pyysalo, Sampo and Laippala, Veronika", editor = "Calzolari, Nicoletta and Kan, Min-Yen and Hoste, Veronique and Lenci, Alessandro and Sakti, Sakriani and Xue, Nianwen", booktitle = "Proceedings of the 2024 Joint International Conference on Computational Linguistics, Language Resources and Evaluation (LREC-COLING 2024)", month = may, year = "2024", address = "Torino, Italia", publisher = "ELRA and ICCL", url = "https://aclanthology.org/2024.lrec-main.234", pages = "2595--2611", abstract = "This article introduces a resource-efficient method for developing question-answer (QA) datasets by extracting QA pairs from web-scale data using machine learning (ML). Our method benefits from recent advances in web register (genre) identification and consists of two ML steps with an additional post-processing step. First, using XLM-R and the multilingual CORE web register corpus series with categories such as QA Forum, we train a multilingual classifier to retrieve documents that are likely to contain QA pairs from web-scale data. Second, we develop a NER-style token classifier to identify the QA text spans within these documents. To this end, we experiment with training on a semi-synthetic dataset built on top of the English LFQA, a small set of manually cleaned web QA pairs in English and Finnish, and a Finnish web QA pair dataset cleaned using ChatGPT. The evaluation of our pipeline demonstrates its capability to efficiently retrieve a substantial volume of QA pairs. While the approach is adaptable to any language given the availability of language models and extensive web data, we showcase its efficiency in English and Finnish, developing the first open, non-synthetic and non-machine translated QA dataset for Finnish {--} Turku WebQA {--} comprising over 200,000 QA pairs.", }
