koki0702/zero-llm-data

Zero LLM数据集提供了从多个公共数据集中提取的经过清理和标准化的文本语料库，专门为书籍《从零开始深度学习❻——LLM篇》准备。每个语料库都组织在自己的目录中（如codebot/、storybot/、webbot/），以便文本数据、BPE编码数据和合并规则分组在一起。这使得数据集易于理解、导航和复现。数据集包括Python代码、TinyStories V2和OpenWebText样本，提供原始文本和BPE编码的token ID。数据集旨在用于小型LLM的从头训练、书籍中的实践练习、分词器训练和实验以及研究和教育目的，不适用于大规模工业LLM训练。

The Zero LLM Dataset provides cleaned, standardized text corpora derived from several public datasets, prepared specifically for use in the book Zero to Deep Learning — LLM Edition. Each corpus is organized into its own directory (e.g., codebot/, storybot/, webbot/) so that text data, BPE-encoded data, and merge rules are grouped together, making the dataset easy to understand, navigate, and reproduce. The dataset includes Python codes, TinyStories V2, and OpenWebText samples, provided as raw text and BPE-encoded token IDs. It is designed for training small LLMs from scratch, hands-on exercises in the book, tokenizer training and experimentation, and research and educational purposes, not intended for large-scale industrial LLM training.