LM-Polygraph/babi_qa

--- language: - en dataset_info: config_name: continuation features: - name: input dtype: string - name: output dtype: string splits: - name: train num_bytes: 8140882 num_examples: 10000 - name: test num_bytes: 814511 num_examples: 1000 download_size: 767405 dataset_size: 8955393 configs: - config_name: continuation data_files: - split: train path: continuation/train-* - split: test path: continuation/test-* --- # Dataset Card for babi_qa <!-- Provide a quick summary of the dataset. --> This is a preprocessed version of babi_qa dataset for benchmarks in LM-Polygraph. ## Dataset Details ### Dataset Description <!-- Provide a longer summary of what this dataset is. --> - **Curated by:** https://huggingface.co/LM-Polygraph - **License:** https://github.com/IINemo/lm-polygraph/blob/main/LICENSE.md ### Dataset Sources [optional] <!-- Provide the basic links for the dataset. --> - **Repository:** https://github.com/IINemo/lm-polygraph ## Uses <!-- Address questions around how the dataset is intended to be used. --> ### Direct Use <!-- This section describes suitable use cases for the dataset. --> This dataset should be used for performing benchmarks on LM-polygraph. ### Out-of-Scope Use <!-- This section addresses misuse, malicious use, and uses that the dataset will not work well for. --> This dataset should not be used for further dataset preprocessing. ## Dataset Structure <!-- This section provides a description of the dataset fields, and additional information about the dataset structure such as criteria used to create the splits, relationships between data points, etc. --> This dataset contains the "continuation" subset, which corresponds to main dataset, used in LM-Polygraph. It may also contain other subsets, which correspond to instruct methods, used in LM-Polygraph. Each subset contains two splits: train and test. Each split contains two string columns: "input", which corresponds to processed input for LM-Polygraph, and "output", which corresponds to processed output for LM-Polygraph. ## Dataset Creation ### Curation Rationale <!-- Motivation for the creation of this dataset. --> This dataset is created in order to separate dataset creation code from benchmarking code. ### Source Data <!-- This section describes the source data (e.g. news text and headlines, social media posts, translated sentences, ...). --> #### Data Collection and Processing <!-- This section describes the data collection and processing process such as data selection criteria, filtering and normalization methods, tools and libraries used, etc. --> Data is collected from https://huggingface.co/datasets/facebook/babi_qa and processed by using build_dataset.py script in repository. #### Who are the source data producers? <!-- This section describes the people or systems who originally created the data. It should also include self-reported demographic or identity information for the source data creators if this information is available. --> People who created https://huggingface.co/datasets/facebook/babi_qa ## Bias, Risks, and Limitations <!-- This section is meant to convey both technical and sociotechnical limitations. --> This dataset contains the same biases, risks, and limitations as its source dataset https://huggingface.co/datasets/facebook/babi_qa ### Recommendations <!-- This section is meant to convey recommendations with respect to the bias, risk, and technical limitations. --> Users should be made aware of the risks, biases and limitations of the dataset.--- 语言： - 英语 数据集信息： 配置名称：延续（continuation） 特征： - 名称：input（输入） 数据类型：字符串 - 名称：output（输出） 数据类型：字符串 划分： - 名称：训练集（train） 字节数：8140882 样本数：10000 - 名称：测试集（test） 字节数：814511 样本数：1000 下载大小：767405 数据集总大小：8955393 配置集： - 配置名称：延续（continuation） 数据文件： - 划分：训练集（train） 路径：continuation/train-* - 划分：测试集（test） 路径：continuation/test-* --- # babi_qa 数据集卡片 本数据集为针对LM-Polygraph（LM-Polygraph）基准测试所预处理的babi问答数据集（babi_qa）版本。 ## 数据集详情 ### 数据集概述 - **整理方：** https://huggingface.co/LM-Polygraph - **授权协议：** https://github.com/IINemo/lm-polygraph/blob/main/LICENSE.md ## 数据集来源[可选] - **代码仓库：** https://github.com/IINemo/lm-polygraph ## 数据集用途 ### 直接用途 本数据集仅可用于在LM-Polygraph上开展基准测试工作。 ### 超出范围的使用场景 本数据集不可用于后续的数据集预处理工作。 ## 数据集结构 本数据集包含对应LM-Polygraph所使用的主数据集的「延续（continuation）」子集，同时也可能包含对应LM-Polygraph所使用的指令微调方法的其他子集。每个子集均包含训练与测试两个划分。每个划分均包含两个字符串类型列：input与output，其中前者对应为LM-Polygraph处理后的输入数据，后者对应为LM-Polygraph处理后的输出数据。 ## 数据集构建 ### 构建初衷 本数据集的构建目的是将数据集构建代码与基准测试代码进行解耦。 ### 源数据 #### 数据收集与处理流程 本数据集的数据源自https://huggingface.co/datasets/facebook/babi_qa，并通过代码仓库中的build_dataset.py脚本完成预处理。 #### 源数据创作者 即https://huggingface.co/datasets/facebook/babi_qa的创建者。 ## 偏差、风险与局限性 本数据集与其源数据集https://huggingface.co/datasets/facebook/babi_qa存在相同的偏差、风险与局限性。 ### 建议 使用者应充分知晓本数据集存在的风险、偏差与局限性。