BookCorpus

该数据集包含了7000本小说，总计约4500万有序句子，旨在用于训练基于循环神经网络（RNN）的模型，以学习高质量的句子表示。其规模达到了4500万句子，任务专注于为学习句子表示训练模型。

This dataset contains 7,000 novels, totaling approximately 45 million ordered sentences. It is intended for training Recurrent Neural Network (RNN)-based models to learn high-quality sentence representations. Given its scale of 45 million sentences, the core task of this dataset is to train models for sentence representation learning.

搜集汇总

背景与挑战

背景概述

BookCorpus是一个大型文本语料库，主要用于无监督学习，特别是句子编码器/解码器的训练。该数据集最初来源于smashwords.com，但已不再公开分发，该仓库提供了爬取和生成类似数据集的脚本和工具。