MLNavigator/russian-retrieval

---
license: mit
---

Based on Sberquad

- Answer converted to human affordable answer.

- Context augmented with some pices of texts from wiki accordant to text on tematic and keywords.

- This dataset cold be used for training retrieval LLM models or modificators for ability of LLM to retrieve target information from collection of tematic related texts.

- Dataset has version with SOURCE data for generating answer with specifing source document for right answer. See file retrieval_dataset_src.jsonl

Dataset consists of 45278 examples in russian language of format:

{
'text': 'text with correct answer',
'q': 'question text',
'a': 'correct answer text',
'context': 'text of 4-10 text chunks, one with right answer and others relevant with text and question on tematic and keywords'
}

Length of one example of context + question + answer is less than 7000 symbols. It should be less than 2048 tokens of rugpt tokenizer.

File retrieval_dataset_src.jsonl has additionally SOURCE data for every text chunk in context, also SOURCE of right answer is set in answer.
This variant of dataset is useful if you need extract answer with specifing source of the right answer.

{
'text': 'text with correct answer',
'q': 'question text',
'a': 'correct answer text with SOURCE data of text',
'context': 'text of 4-10 text chunks, one with right answer and others relevant with text and question on tematic and keywords.
Each of text chunks has it's own SOURCE data'
}

All SOURCE data are sintetic generated and not real.

原始信息汇总

数据集概述

数据集描述

• 该数据集基于Sberquad，旨在提供人类可理解的答案。
• 上下文通过添加与主题和关键词相关的维基文本片段进行增强。
• 适用于训练检索型大型语言模型（LLM）或改进LLM从相关主题文本集合中检索目标信息的能力。

数据格式

• 数据集包含45278个俄语示例，格式如下： json { text: 包含正确答案的文本, q: 问题文本, a: 正确答案文本, context: 包含4-10个文本片段的上下文，其中一个包含正确答案，其他与文本和问题主题和关键词相关 }

• 每个示例的上下文、问题和答案的总长度小于7000个字符，应小于2048个rugpt分词器的标记。

特殊版本

• 文件retrieval_dataset_src.jsonl包含每个上下文文本片段的SOURCE数据，以及正确答案的SOURCE数据。 json { text: 包含正确答案的文本, q: 问题文本, a: 包含正确答案及其SOURCE数据的文本, context: 包含4-10个文本片段的上下文，每个文本片段都有其SOURCE数据 }

• 所有SOURCE数据均为合成生成，并非真实数据。