timaeus/pile-github

该数据集包含文本和元数据两个主要特征。文本特征为字符串类型，元数据特征包含一个名为pile_set_name的字符串类型字段。数据集包含一个训练分割，共有100,000个样本，占用542,456,309字节。数据集的下载大小为202,975,613字节，总大小为542,456,309字节。默认配置指定数据文件路径为data/train-*。

The dataset contains two main features: text and meta. The text feature is of string type, and the meta feature is a structure containing a string field named pile_set_name. The dataset includes a train split with 100,000 examples, occupying 542,456,309 bytes. The download size of the dataset is 202,975,613 bytes, and the total size is 542,456,309 bytes. The default configuration specifies the data file path as data/train-*.