BookTest

BookTest数据集由IBM沃森研究中心创建，是一个专为阅读理解任务设计的大型数据集，其规模是流行的Children’s Book Test数据集的60倍以上。该数据集源自Project Gutenberg的版权免费书籍，通过使用Stanford POS和NER工具进行实体和名词的识别，生成包含缺失词的句子作为训练样本。BookTest数据集不仅支持训练更大规模的模型，而且其创建方法可扩展至更大规模的数据集。该数据集的应用领域主要集中在提高机器对自然语言的理解能力，特别是在回答依赖于上下文文档理解的封闭式问题方面。

The BookTest dataset was created by IBM Watson Research Center. It is a large-scale dataset specifically designed for reading comprehension tasks, with a scale exceeding 60 times that of the widely used Children’s Book Test dataset. Sourced from copyright-free books on Project Gutenberg, it generates training samples in the form of sentences with missing words by identifying entities and nouns via Stanford POS and NER tools. The BookTest dataset not only supports training larger-scale models, but its construction methodology is also scalable to even larger datasets. Its primary application focuses on advancing machine natural language understanding, particularly in answering closed-ended questions that rely on contextual document comprehension.