visionscaper/agentic-llm-pretraining-1.7b

--- license: cc-by-sa-4.0 language: - en size_categories: - 1M<n<10M task_categories: - text-generation - question-answering tags: - pretraining - agentic - function-calling - reasoning - code - math - dialogue - rag pretty_name: Agentic LLM Pretraining Dataset (1.7B tokens) --- # Agentic LLM Pretraining Dataset A pretraining corpus for small language models (1-3B parameters) optimized for agentic tasks. The corpus emphasizes learning to comprehend language, reason, follow instructions, and use tools over memorizing factual knowledge — the assumption is that domain knowledge will be provided at runtime via RAG. The idea is that this could enable much smaller pretraining corpora by omitting the large volumes of text typically needed to memorize facts. ## Dataset Summary | Split | Samples | Estimated Tokens | Purpose | |-------|---------|------------------|---------| | Train | 2,655,695 | ~1.19B | Model training | | Validation | 569,077 | ~255M | Monitor generalization during training | | Test | 569,078 | ~255M | Final model evaluation | | **Total** | **3,793,850** | **~1.7B** | | ## Intended Use - **Primary**: Pretraining language models (1-3B parameters) for agentic applications - **Secondary**: Fine-tuning, evaluation, or research on multi-task learning ## Dataset Composition The dataset combines 27 source datasets across 8 categories: | Category | Datasets | Samples | Description | |----------|----------|---------|-------------| | Grammar/Language | fineweb-edu, fineweb-edu-long, wikipedia, simple-wikipedia | 585,446 | High-quality educational and encyclopedic text | | Reasoning/Math | openmath-instruct-1, openmath-instruct-2, gsm8k, math, ecqa | 772,308 | Mathematical reasoning with step-by-step solutions | | Procedural | cosmopedia-wikihow, stackexchange | 335,594 | How-to articles and technical Q&A | | Code | swe-bench, code-contests, codesearchnet | 442,273 | Programming tasks, debugging, and code documentation | | Agentic | toolace, hermes-function-calling, glaive-function-calling | 126,153 | Function calling with tool use patterns | | Knowledge | generics-kb, openbookqa | 1,025,825 | Factual knowledge and science reasoning | | Dialogue | soda, multiwoz, wizard-of-wikipedia, sharegpt, empathetic-dialogues, samsum | 480,329 | Conversational data with various styles | | RAG | rag-dataset-12000, ragbench-hotpotqa | 11,481 | Retrieval-augmented generation patterns | ### Per-Dataset Details | Dataset | Total | License | HuggingFace Source | Filter | Quality* | |---------|-------|---------|-------------------|--------|---------| | generics-kb | 1,020,868 | CC BY 4.0 | [generics_kb](https://huggingface.co/datasets/generics_kb) | score > 0.5 | Medium | | openmath-instruct-1 | 482,677 | NVIDIA License | [nvidia/OpenMathInstruct-1](https://huggingface.co/datasets/nvidia/OpenMathInstruct-1) | Correct solutions only | Good | | codesearchnet | 412,178 | Various | [claudios/code_search_net](https://huggingface.co/datasets/claudios/code_search_net) | — | Good | | soda | 359,329 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio) (SODA) | — | Good | | wikipedia | 309,800 | CC BY-SA 3.0 | [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) (20231101.en) | — | Good | | openmath-instruct-2 | 262,060 | CC BY 4.0 | [nvidia/OpenMathInstruct-2](https://huggingface.co/datasets/nvidia/OpenMathInstruct-2) | — | Good | | simple-wikipedia | 241,787 | CC BY-SA 3.0 | [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia) (20231101.simple) | — | Medium | | cosmopedia-wikihow | 179,000 | Apache 2.0 | [HuggingFaceTB/cosmopedia](https://huggingface.co/datasets/HuggingFaceTB/cosmopedia) (wikihow) | — | Good | | stackexchange | 156,594 | CC BY-SA 4.0 | [HuggingFaceH4/stack-exchange-preferences](https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences) | — | Good | | glaive-function-calling | 112,960 | Apache 2.0 | [glaiveai/glaive-function-calling-v2](https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2) | — | Good | | sharegpt | 76,041 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio) (ShareGPT) | — | Good | | fineweb-edu | 28,765 | ODC-By | [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) (sample-10BT) | — | Good | | swe-bench | 19,000 | MIT | [princeton-nlp/SWE-bench](https://huggingface.co/datasets/princeton-nlp/SWE-bench) | — | Good | | wizard-of-wikipedia | 18,430 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio) (wizard_of_wikipedia) | — | Good | | empathetic-dialogues | 17,802 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio) (Empathetic) | — | Low | | samsum | 14,731 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio) (SAMSum) | — | Good | | math | 12,500 | MIT | [qwedsacf/competition_math](https://huggingface.co/datasets/qwedsacf/competition_math) | — | Good | | code-contests | 11,095 | CC BY 4.0 | [deepmind/code_contests](https://huggingface.co/datasets/deepmind/code_contests) | Has C++/Python3/Java solution | Good | | toolace | 11,300 | Apache 2.0 | [Team-ACE/ToolACE](https://huggingface.co/datasets/Team-ACE/ToolACE) | — | Good | | rag-dataset-12000 | 9,598 | Apache 2.0 | [neural-bridge/rag-dataset-12000](https://huggingface.co/datasets/neural-bridge/rag-dataset-12000) | — | Good | | multiwoz | 8,437 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio) (MULTIWOZ2_2) | — | Medium | | ecqa | 7,598 | CDLA-Sharing-1.0 | [tasksource/ecqa](https://huggingface.co/datasets/tasksource/ecqa) | — | Medium | | gsm8k | 7,473 | MIT | [openai/gsm8k](https://huggingface.co/datasets/openai/gsm8k) | — | Good | | fineweb-edu-long | 5,094 | ODC-By | [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu) (sample-100BT) | ≥120K chars, quality ≥3 | Good | | openbookqa | 4,957 | Apache 2.0 | [allenai/openbookqa](https://huggingface.co/datasets/allenai/openbookqa) | — | Good | | ragbench-hotpotqa | 1,883 | CC BY 4.0 | [rungalileo/ragbench](https://huggingface.co/datasets/rungalileo/ragbench) (hotpotqa) | — | Good | | hermes-function-calling | 1,893 | Apache 2.0 | [NousResearch/hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1) | — | Good | *Quality as assessed through manual inspection of a small number of samples: **Good** = high-quality, appropriate for training; **Medium** = some issues but acceptable; **Low** = quality concerns but kept for coverage. ## Data Preprocessing and Formatting Each sample is a JSON object with three fields: ```json {"source": "gsm8k", "index": 42, "text": "Problem: ..."} ``` - `source`: Original dataset name - `index`: Sample index within the source dataset - `text`: Preprocessed text ready for tokenization ### Text Formats by Category **Free text** (output as-is): - Wikipedia articles with markdown headings - Code functions with docstrings - Short factual statements **Structured reasoning** (Problem/Solution format): ``` Problem: A store sells apples for $2 each... Solution: Let's solve this step by step. 1. First, calculate the total cost... ... The answer is \boxed{42}. ``` **Conversational** (Qwen3 chat format): ``` <|im_start|>system You are a helpful assistant.<|im_end|> <|im_start|>user How do I sort a list in Python?<|im_end|> <|im_start|>assistant You can use the sorted() function...<|im_end|> ``` **Function calling** (formats vary by source dataset): - `<functioncall>{"name": "func", "arguments": {...}}</functioncall>` (glaive-function-calling) - `<tool_call>{"name": "func", ...}</tool_call>` (hermes-function-calling) - `[func(param=value)]` (toolace) ### Preprocessing Steps All samples were preprocessed with dataset-specific transformations: 1. **Filtering**: Quality and other property filters applied per-dataset (see Filter column in table above) 2. **Preprocessing**: Sample data fields extracted and normalized per dataset 3. **Templating**: Preprocessed fields applied to text templates (e.g., Qwen3 chat format for conversational datasets) 4. **Global shuffling**: Samples randomly shuffled across all sources ### Source Code For full details on data acquisition, filtering, and preprocessing, see the code [here](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/), specifically: - [training_metadata.json](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/training_metadata.json) — Acquisition, filtering, and preprocessing metadata per dataset - [preprocessing.py](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/preprocessing.py) — Sample preprocessing functions per dataset - [dataset_templates.py](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/dataset_templates.py) — Templates for representing preprocessed data as text - [create_splits.py](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/create_splits.py) — Global shuffling and train/val/test split creation ## Usage ### Load the dataset ```python from datasets import load_dataset # Load all splits dataset = load_dataset("visionscaper/agentic-llm-pretraining-1.7b") # Load specific split train = load_dataset("visionscaper/agentic-llm-pretraining-1.7b", split="train") ``` ### Filter by source ```python # Get only math reasoning samples math_data = train.filter(lambda x: x["source"] in ["gsm8k", "math", "openmath-instruct-1"]) # Get only dialogue samples dialogue_sources = ["soda", "sharegpt", "wizard-of-wikipedia", "multiwoz"] dialogue_data = train.filter(lambda x: x["source"] in dialogue_sources) # Get only agentic/function-calling samples agentic_sources = ["toolace", "hermes-function-calling", "glaive-function-calling"] agentic_data = train.filter(lambda x: x["source"] in agentic_sources) ``` ### Tokenization example ```python from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B") def tokenize(example): return tokenizer(example["text"], truncation=True, max_length=32768) tokenized = train.map(tokenize, batched=True, remove_columns=["source", "index", "text"]) ``` ## Known Limitations - **fineweb-edu**: Quality score filtering (`int_score >= 3`) was not applied. All quality levels (1-5) are included. Should be fixed in v2. - **empathetic-dialogues**: Quality is low (role confusion, peer-to-peer chat style). Consider an alternative in the future. ## License This dataset is released under **CC BY-SA 4.0** (Creative Commons Attribution-ShareAlike 4.0 International). You are free to: - **Share** — copy and redistribute the material - **Adapt** — remix, transform, and build upon the material for any purpose, including commercial Under the following terms: - **Attribution** — You must give appropriate credit and indicate if changes were made - **ShareAlike** — If you remix or transform the material, you must distribute under the same license Individual samples retain their original source licenses. All source datasets permit commercial use. See the per-dataset license column in the table above for details. ## Citation ```bibtex @dataset{snijder2025agentic, author = {Snijder, Freddy}, title = {Agentic LLM Pretraining Dataset}, year = {2025}, publisher = {HuggingFace}, url = {https://huggingface.co/datasets/visionscaper/agentic-llm-pretraining-1.7b} } ``` If you use this dataset, please also cite the original source datasets appropriately. ## Contact Created by [Visionscaper](https://github.com/visionscaper) for the Minimal NTP → RLP research project.许可证：CC BY-SA 4.0（知识共享署名-相同方式共享4.0国际协议） 语言：英语 规模类别：100万 < 样本数 < 1000万 任务类别：文本生成、问答 标签：预训练、智能体、函数调用、推理、代码、数学、对话、检索增强生成（RAG） 美观名称：智能体大语言模型预训练数据集（17亿Token） # 智能体大语言模型预训练数据集 本数据集为面向智能体任务优化的小型语言模型（10亿~30亿参数）构建的预训练语料库。相较于记忆事实性知识，该语料库更侧重语言理解、推理、指令遵循与工具使用能力——其核心假设为领域知识可在推理阶段通过检索增强生成（RAG）获取。本数据集通过省略通常用于记忆事实的大量文本，得以使用规模更小的预训练语料库。 ## 数据集摘要 | 划分 | 样本数 | 预估Token数 | 用途 | |-------|---------|------------------|---------| | 训练集 | 2,655,695 | ~11.9亿 | 模型训练 | | 验证集 | 569,077 | ~2.55亿 | 训练过程中监控泛化能力 | | 测试集 | 569,078 | ~2.55亿 | 最终模型评估 | | **总计** | **3,793,850** | **~17亿** | | ## 预期用途 - **主要用途**：为智能体应用预训练（10亿~30亿参数）语言模型 - **次要用途**：多任务学习的微调、评估或研究 ## 数据集组成 本数据集整合了8个类别共27个源数据集： | 类别 | 源数据集 | 样本数 | 描述 | |----------|----------|---------|-------------| | 语法/语言 | fineweb-edu、fineweb-edu-long、wikipedia、simple-wikipedia | 585,446 | 高质量教育与百科文本 | | 推理/数学 | openmath-instruct-1、openmath-instruct-2、gsm8k、math、ecqa | 772,308 | 包含分步解答的数学推理数据 | | 操作指南 | cosmopedia-wikihow、stackexchange | 335,594 | 教程文章与技术问答 | | 代码 | swe-bench、code-contests、codesearchnet | 442,273 | 编程任务、调试与代码文档 | | 智能体 | toolace、hermes-function-calling、glaive-function-calling | 126,153 | 包含工具使用模式的函数调用数据 | | 知识 | generics-kb、openbookqa | 1,025,825 | 事实性知识与科学推理数据 | | 对话 | soda、multiwoz、wizard-of-wikipedia、sharegpt、empathetic-dialogues、samsum | 480,329 | 多种风格的对话数据 | | 检索增强生成 | rag-dataset-12000、ragbench-hotpotqa | 11,481 | 检索增强生成模式数据 | ### 单数据集详情 | 数据集 | 总样本数 | 许可证 | HuggingFace源地址 | 过滤规则 | 质量评分* | |---------|-------|---------|-------------------|--------|---------| | generics-kb | 1,020,868 | CC BY 4.0 | [generics_kb](https://huggingface.co/datasets/generics_kb) | 得分 > 0.5 | 中等 | | openmath-instruct-1 | 482,677 | NVIDIA许可证 | [nvidia/OpenMathInstruct-1](https://huggingface.co/datasets/nvidia/OpenMathInstruct-1) | 仅保留正确解答 | 优秀 | | codesearchnet | 412,178 | 多许可证 | [claudios/code_search_net](https://huggingface.co/datasets/claudios/code_search_net) | 无 | 优秀 | | soda | 359,329 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio)（SODA） | 无 | 优秀 | | wikipedia | 309,800 | CC BY-SA 3.0 | [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia)（20231101.en） | 无 | 优秀 | | openmath-instruct-2 | 262,060 | CC BY 4.0 | [nvidia/OpenMathInstruct-2](https://huggingface.co/datasets/nvidia/OpenMathInstruct-2) | 无 | 优秀 | | simple-wikipedia | 241,787 | CC BY-SA 3.0 | [wikimedia/wikipedia](https://huggingface.co/datasets/wikimedia/wikipedia)（20231101.simple） | 无 | 中等 | | cosmopedia-wikihow | 179,000 | Apache 2.0 | [HuggingFaceTB/cosmopedia](https://huggingface.co/datasets/HuggingFaceTB/cosmopedia)（wikihow） | 无 | 优秀 | | stackexchange | 156,594 | CC BY-SA 4.0 | [HuggingFaceH4/stack-exchange-preferences](https://huggingface.co/datasets/HuggingFaceH4/stack-exchange-preferences) | 无 | 优秀 | | glaive-function-calling | 112,960 | Apache 2.0 | [glaiveai/glaive-function-calling-v2](https://huggingface.co/datasets/glaiveai/glaive-function-calling-v2) | 无 | 优秀 | | sharegpt | 76,041 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio)（ShareGPT） | 无 | 优秀 | | fineweb-edu | 28,765 | ODC-By | [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu)（sample-10BT） | 无 | 优秀 | | swe-bench | 19,000 | MIT许可证 | [princeton-nlp/SWE-bench](https://huggingface.co/datasets/princeton-nlp/SWE-bench) | 无 | 优秀 | | wizard-of-wikipedia | 18,430 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio)（wizard_of_wikipedia） | 无 | 优秀 | | empathetic-dialogues | 17,802 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio)（Empathetic） | 无 | 低 | | samsum | 14,731 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio)（SAMSum） | 无 | 优秀 | | math | 12,500 | MIT许可证 | [qwedsacf/competition_math](https://huggingface.co/datasets/qwedsacf/competition_math) | 无 | 优秀 | | code-contests | 11,095 | CC BY 4.0 | [deepmind/code_contests](https://huggingface.co/datasets/deepmind/code_contests) | 包含C++/Python3/Java解答 | 优秀 | | toolace | 11,300 | Apache 2.0 | [Team-ACE/ToolACE](https://huggingface.co/datasets/Team-ACE/ToolACE) | 无 | 优秀 | | rag-dataset-12000 | 9,598 | Apache 2.0 | [neural-bridge/rag-dataset-12000](https://huggingface.co/datasets/neural-bridge/rag-dataset-12000) | 无 | 优秀 | | multiwoz | 8,437 | Apache 2.0 | [Salesforce/dialogstudio](https://huggingface.co/datasets/Salesforce/dialogstudio)（MULTIWOZ2_2） | 无 | 中等 | | ecqa | 7,598 | CDLA-Sharing-1.0 | [tasksource/ecqa](https://huggingface.co/datasets/tasksource/ecqa) | 无 | 中等 | | gsm8k | 7,473 | MIT许可证 | [openai/gsm8k](https://huggingface.co/datasets/openai/gsm8k) | 无 | 优秀 | | fineweb-edu-long | 5,094 | ODC-By | [HuggingFaceFW/fineweb-edu](https://huggingface.co/datasets/HuggingFaceFW/fineweb-edu)（sample-100BT） | 字符数 ≥12万，质量评分 ≥3 | 优秀 | | openbookqa | 4,957 | Apache 2.0 | [allenai/openbookqa](https://huggingface.co/datasets/allenai/openbookqa) | 无 | 优秀 | | ragbench-hotpotqa | 1,883 | CC BY 4.0 | [rungalileo/ragbench](https://huggingface.co/datasets/rungalileo/ragbench)（hotpotqa） | 无 | 优秀 | | hermes-function-calling | 1,893 | Apache 2.0 | [NousResearch/hermes-function-calling-v1](https://huggingface.co/datasets/NousResearch/hermes-function-calling-v1) | 无 | 优秀 | *质量评分通过少量样本人工评估得出：**优秀**=高质量，适合训练；**中等**=存在少量问题但可接受；**低**=存在质量隐患但为覆盖范围保留。 ## 数据预处理与格式 每个样本为包含三个字段的JSON对象： json {"source": "gsm8k", "index": 42, "text": "问题：..."} - `source`：原始数据集名称 - `index`：该样本在源数据集中的索引 - `text`：已完成预处理、可直接用于Token化的文本 ### 按类别划分的文本格式 **自由文本**（直接输出）： - 带Markdown标题的维基百科文章 - 带文档字符串的代码函数 - 简短事实陈述 **结构化推理**（问题/解决方案格式）： 问题：某商店苹果售价为每个2美元... 解决方案：我们逐步解决这个问题。 1. 首先，计算总成本... ... 答案是 oxed{42}。 **对话格式**（采用Qwen3对话格式）： <|im_start|>system 你是一位乐于助人的助手。<|im_end|> <|im_start|>user 如何在Python中对列表进行排序？<|im_end|> <|im_start|>assistant 你可以使用sorted()函数...<|im_end|> **函数调用**（格式随源数据集不同有所差异）： - `<functioncall>{"name": "func", "arguments": {...}}</functioncall>`（glaive-function-calling） - `<tool_call>{"name": "func", ...}</tool_call>`（hermes-function-calling） - `[func(param=value)]`（toolace） ### 预处理步骤 所有样本均通过数据集专属变换完成预处理： 1. **过滤**：按数据集应用质量与其他属性过滤规则（详见上方表格的Filter列） 2. **预处理**：提取并标准化各数据集的样本数据字段 3. **模板化**：将预处理后的字段应用至文本模板（如对话数据集采用Qwen3对话格式） 4. **全局洗牌**：对所有源数据集的样本进行随机打乱 ### 源代码 如需了解数据获取、过滤与预处理的完整细节，请参阅以下代码： - [training_metadata.json](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/training_metadata.json)：各数据集的获取、过滤与预处理元数据 - [preprocessing.py](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/preprocessing.py)：各数据集的样本预处理函数 - [dataset_templates.py](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/dataset_templates.py)：将预处理后的数据转换为文本的模板 - [create_splits.py](https://github.com/nuhame/mlpug/blob/feature/pretraining-agentic-llm-example-29122025/examples/agentic_llm_pretraining/datasets/create_splits.py)：全局洗牌与训练/验证/测试集划分代码 ## 使用方法 ### 加载数据集 python from datasets import load_dataset # 加载所有划分 dataset = load_dataset("visionscaper/agentic-llm-pretraining-1.7b") # 加载指定划分 train = load_dataset("visionscaper/agentic-llm-pretraining-1.7b", split="train") ### 按源数据集过滤 python # 仅获取数学推理样本 math_data = train.filter(lambda x: x["source"] in ["gsm8k", "math", "openmath-instruct-1"]) # 仅获取对话样本 dialogue_sources = ["soda", "sharegpt", "wizard-of-wikipedia", "multiwoz"] dialogue_data = train.filter(lambda x: x["source"] in dialogue_sources) # 仅获取智能体/函数调用样本 agentic_sources = ["toolace", "hermes-function-calling", "glaive-function-calling"] agentic_data = train.filter(lambda x: x["source"] in agentic_sources) ### Token化示例 python from transformers import AutoTokenizer tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen3-1.7B") def tokenize(example): return tokenizer(example["text"], truncation=True, max_length=32768) tokenized = train.map(tokenize, batched=True, remove_columns=["source", "index", "text"]) ## 已知局限 1. **fineweb-edu**：未应用质量评分过滤规则（`int_score >= 3`），包含所有质量等级（1-5）的样本，该问题将在v2版本修复。 2. **empathetic-dialogues**：质量较低（存在角色混淆、点对点对话风格问题），未来可考虑替换为其他数据集。 ## 许可证 本数据集采用**CC BY-SA 4.0**（知识共享署名-相同方式共享4.0国际协议）发布。 您可以自由： - **共享**：复制并重新分发本材料 - **改编**：重新混合、转换并基于本材料创作任何用途的作品，包括商业用途 需遵守以下条款： - **署名**：您必须给出适当的署名并说明是否对材料进行了修改 - **相同方式共享**：如果您重新混合或转换本材料，必须以相同许可证分发衍生作品 单个样本保留其原始源数据集的许可证，所有源数据集均允许商业使用，详情请参阅上方表格的各数据集许可证列。 ## 引用 bibtex @dataset{snijder2025agentic, author = {Snijder, Freddy}, title = {Agentic LLM Pretraining Dataset}, year = {2025}, publisher = {HuggingFace}, url = {https://huggingface.co/datasets/visionscaper/agentic-llm-pretraining-1.7b} } 如果使用本数据集，请同时适当引用原始源数据集。 ## 联系方式 本数据集由[Visionscaper](https://github.com/visionscaper)为Minimal NTP → RLP研究项目创建。