moqarashad/fineweb-edu-ar-en-streamed

FineWeb EDU AR+EN是一个包含阿拉伯语和英语的多语言文本生成数据集。数据通过流式处理并以等概率交替混合两种语言的数据。数据集使用Parquet格式存储，每个压缩后的分片大小约为100MB，并且进行了高效的压缩和随机化处理。FineWeb EDU AR+EN is a multilingual text-generation dataset containing both Arabic and English. The data is streamed and interleaved with equal probability for both languages. The dataset is stored in the Parquet format, with each compressed shard being about 100MB in size, and it has been efficiently compressed and randomized.