jobs-git/Zyda-2

Zyda-2是一个包含超过5万亿个token的语言模型数据集，由收集开源和高质量数据集、合并后进行去重和基于模型的过滤质量处理而创建。Zyda-2包含了多样化的网络数据源、高度教育性的内容、数学、代码和科学论文。

Zyda-2 is a language modeling dataset with over 5 trillion tokens, created by collecting open and high-quality datasets, merging them, and then performing cross-deduplication and model-based quality filtering. Zyda-2 comprises diverse sources of web data, highly educational content, math, code, and scientific papers.