nwdxlgzs/sentence-pt-enzh-tr

这个数据集包含了中文和英文的预训练数据，其中中文数据总行数为434,818,374，英文数据总行数为450,173,542。中文比例约为49.13%，英文比例约为50.87%。数据集还提供了一个中英混合的版本，每个文件都包含中英文数据，且尽可能保持相同的比例。不过，处理程序在处理中英混合版本时遗漏了最后几批数据，因此只有210个文件，每个文件大约4MB，总大小约为69GB。

This dataset includes pre-trained data in both Chinese and English, with a total of 434,818,374 Chinese lines and 450,173,542 English lines. The Chinese proportion is about 49.13%, and the English proportion is about 50.87%. There is also a mixed version of the dataset that contains both Chinese and English in each file, maintaining the proportions as much as possible. However, the processing program missed the last few batches of data when handling the mixed version, resulting in only 210 files, each about 4MB in size, totaling about 69GB.