smartcat/natural_quesions_sr

--- license: mit task_categories: - question-answering language: - sr pretty_name: Natural Questions Serbian size_categories: - 10K<n<100K --- # Dataset Card for Serbian Natural Questions (Subset) ## Dataset Description - **Repository:** [Hugging Face Dataset](https://huggingface.co/datasets/smartcat/natural_questions_sr) - **Point of Contact:** [SmartCat.io] ### Dataset Summary This dataset is a Serbian translation of the first 8,000 examples from Google's Natural Questions (NQ) dataset. It contains real user questions and corresponding Wikipedia articles, automatically translated from English to Serbian. The dataset is designed for evaluating embedding models on Question Answering (QA) and Information Retrieval (IR) tasks in the Serbian language, offering a more realistic and challenging task compared to prior QA datasets. The original Natural Questions dataset can be retrieved from: https://huggingface.co/datasets/google-research-datasets/natural_questions ### Supported Tasks and Leaderboards - **Question Answering**: The dataset can be used to evaluate models' ability to answer questions in Serbian based on given Wikipedia articles. - **Information Retrieval**: It can also be used to assess models' performance in retrieving relevant information from a corpus of Serbian text. - **Reading Comprehension**: The dataset challenges models to comprehend entire Wikipedia articles to find answers, mimicking real-world information seeking scenarios. ### Languages The dataset is in Serbian (sr). ## Dataset Structure ### Data Instances Each instance in the dataset contains: - `id`: The original Natural Questions question ID - `question`: The question translated to Serbian - `article`: The corresponding Wikipedia article translated to Serbian ### Data Fields - `id`: string - `question`: string - `article`: string ### Data Splits The dataset consists of 8,000 examples from the original Natural Questions dataset. There are no predefined train/validation/test splits. ## Dataset Creation ### Curation Rationale This dataset was created to provide a resource for evaluating NLP models on Serbian language tasks, particularly in the domains of question answering, information retrieval, and reading comprehension. The use of real user questions and full Wikipedia articles makes it a more challenging and realistic task. ### Source Data #### Initial Data Collection and Normalization The source data is derived from Google's Natural Questions dataset, which contains real user questions and corresponding Wikipedia articles. #### Who are the source language producers? The original questions were posed by real users on Google. The answers and articles are from Wikipedia. ### Annotations #### Annotation process The original English dataset was automatically translated to Serbian using the GPT-3.5-Turbo-0125 model. #### Who are the annotators? The translation was performed automatically by an AI model, without human intervention. ### Personal and Sensitive Information The dataset may contain personal information present in the original Natural Questions dataset. Users should be aware of this and handle the data accordingly. ## Considerations for Using the Data ### Social Impact of Dataset This dataset contributes to the development of NLP technologies for the Serbian language, potentially improving access to information and language technologies for Serbian speakers. It also promotes the development of more robust QA systems that can handle real-world queries. ### Discussion of Biases The dataset may inherit biases present in the original Natural Questions dataset, including biases in question selection and Wikipedia content. Additionally, the automatic translation process may introduce its own biases or errors. ### Other Known Limitations - The quality of the Serbian translations has been manually verified, but may contain errors. - The dataset is limited to the first 8,000 examples of Natural Questions, which may not be fully representative of the entire dataset. - The complexity of translating entire Wikipedia articles may lead to inconsistencies or loss of context in some cases. ## Additional Information ### Dataset Curators [SmartCat.io] ### Licensing Information [MIT] ### Citation Information If you use this dataset, please cite both the original Natural Questions dataset and this Serbian translation: ``` @article{kwiatkowski2019natural, title={Natural questions: a benchmark for question answering research}, author={Kwiatkowski, Tom and Palomaki, Jennimaria and Redfield, Olivia and Collins, Michael and Parikh, Ankur and Alberti, Chris and Epstein, Danielle and Polosukhin, Illia and Devlin, Jacob and Lee, Kenton and others}, journal={Transactions of the Association for Computational Linguistics}, volume={7}, pages={453--466}, year={2019}, publisher={MIT Press} } @misc{serbian-nq-subset, title={Serbian Natural Questions Subset}, author={[Your Name]}, year={2024}, howpublished={\url{https://huggingface.co/datasets/your-username/serbian-nq-subset}} } ``` ### Contributions Thanks to Google for creating the original Natural Questions dataset. ## Loading the Dataset Here's a Python code example to load the dataset using the Hugging Face `datasets` library: ```python from datasets import load_dataset # Load the dataset dataset = load_dataset("smartcat/natural_questions_sr") # Access the data for example in dataset['train']: print(f"ID: {example['id']}") print(f"Question: {example['question']}") print(f"Article: {example['article'][:100]}...") # Print first 100 characters of the article print("---") ```--- 许可证：MIT协议 任务类别： - 问答（Question Answering） 语言： - 塞尔维亚语（sr） 展示名称：塞尔维亚语自然问答数据集 规模类别： - 10K<n<100K --- # 塞尔维亚语自然问答（子集）数据集卡片 ## 数据集说明 - **仓库地址**：[Hugging Face数据集](https://huggingface.co/datasets/smartcat/natural_questions_sr) - **联系人**：[SmartCat.io] ### 数据集概述 本数据集为谷歌自然问答（Natural Questions, NQ）数据集前8000条样本的塞尔维亚语译版，包含经自动英译塞翻译得到的真实用户提问与对应维基百科文章。本数据集专为评估塞尔维亚语环境下的嵌入模型在问答（Question Answering, QA）与信息检索（Information Retrieval, IR）任务上的性能而设计，相较于此前的问答数据集，能够提供更贴近真实场景且更具挑战性的任务场景。 原始自然问答数据集可通过以下链接获取：https://huggingface.co/datasets/google-research-datasets/natural_questions ### 支持任务与排行榜 - **问答（Question Answering）**：本数据集可用于评估模型基于给定维基百科文章回答塞尔维亚语提问的能力。 - **信息检索（Information Retrieval）**：亦可用于评估模型从塞尔维亚语文本语料库中检索相关信息的性能表现。 - **阅读理解（Reading Comprehension）**：本数据集要求模型通读完整维基百科文章以提取答案，模拟真实世界的信息获取场景。 ### 语言说明 本数据集采用塞尔维亚语（sr）。 ## 数据集结构 ### 数据实例 本数据集的每条数据实例包含以下字段： - `id`：原始自然问答数据集的问题ID - `question`：译为塞尔维亚语的提问文本 - `article`：译为塞尔维亚语的对应维基百科文章 ### 数据字段说明 - `id`：字符串类型（string） - `question`：字符串类型（string） - `article`：字符串类型（string） ### 数据划分 本数据集包含原始自然问答数据集的8000条样本，未预设训练集、验证集与测试集划分。 ## 数据集构建 ### 构建初衷 本数据集旨在为塞尔维亚语自然语言处理（Natural Language Processing, NLP）模型的评估提供支撑资源，尤其聚焦于问答、信息检索与阅读理解三大任务领域。采用真实用户提问与完整维基百科文章的设置，使得该数据集具备更强的挑战性与真实感。 ### 源数据 #### 初始数据收集与标准化 本数据集的源数据源自谷歌自然问答数据集，该数据集包含真实用户提问与对应维基百科文章。 #### 源语言内容创作者 原始提问由谷歌平台上的真实用户提出，答案与文章内容均来自维基百科。 ### 标注信息 #### 标注流程 原始英文数据集通过GPT-3.5-Turbo-0125模型自动翻译为塞尔维亚语。 #### 标注者 本次翻译由AI模型自动完成，未经过人工干预。 ### 个人与敏感信息 本数据集可能包含原始自然问答数据集中原有的个人敏感信息，使用者需知悉该风险并规范处理数据。 ## 数据使用注意事项 ### 数据集社会影响 本数据集有助于推动塞尔维亚语自然语言处理技术的发展，有望提升塞尔维亚语使用者的信息获取能力与语言技术应用水平，同时助力开发能够处理真实场景提问的鲁棒性更强的问答系统。 ### 偏差问题讨论 本数据集可能继承原始自然问答数据集的固有偏差，包括提问选择与维基百科内容中的偏差。此外，自动翻译流程也可能引入额外的偏差或错误。 ### 其他已知局限性 - 塞尔维亚语译文的质量已通过人工核验，但仍可能存在错误。 - 本数据集仅包含自然问答数据集的前8000条样本，无法完全代表完整数据集的分布特征。 - 完整维基百科文章的翻译复杂度较高，部分场景下可能出现上下文不一致或信息丢失的问题。 ## 附加信息 ### 数据集维护者 [SmartCat.io] ### 许可证信息 [MIT协议] ### 引用规范 若使用本数据集，请同时引用原始自然问答数据集与本次塞尔维亚语译版： @article{kwiatkowski2019natural, title={Natural questions: a benchmark for question answering research}, author={Kwiatkowski, Tom and Palomaki, Jennimaria and Redfield, Olivia and Collins, Michael and Parikh, Ankur and Alberti, Chris and Epstein, Danielle and Polosukhin, Illia and Devlin, Jacob and Lee, Kenton and others}, journal={Transactions of the Association for Computational Linguistics}, volume={7}, pages={453--466}, year={2019}, publisher={MIT Press} } @misc{serbian-nq-subset, title={Serbian Natural Questions Subset}, author={[Your Name]}, year={2024}, howpublished={url{https://huggingface.co/datasets/your-username/serbian-nq-subset}} } ### 致谢 感谢谷歌团队创建原始自然问答数据集。 ## 数据集加载 以下为使用Hugging Face `datasets`库加载本数据集的Python代码示例： python from datasets import load_dataset # Load the dataset dataset = load_dataset("smartcat/natural_questions_sr") # Access the data for example in dataset['train']: print(f"ID: {example['id']}") print(f"Question: {example['question']}") print(f"Article: {example['article'][:100]}...") # Print first 100 characters of the article print("---")