LumiVore/lumivore-stage1-training-data

# Lumivore Stage 1 Training Dataset **Version:** 4.5-A **Created:** March 2026 **Purpose:** Full fine-tuning of Qwen2.5-0.5B base model to MoE architecture **Total Examples:** ~11,600 **Format:** Alpaca (instruction, input, output) --- ## Overview This dataset was used for Stage 1 of the Lumivore-1.2B training pipeline. It combines general agentic task data with reasoning examples to teach the base model (Qwen2.5-0.5B) how to perform tool use, reasoning, and structured outputs before MoE conversion. --- ## Data Sources | Source | Description | Proportion | |--------|-------------|------------| | **TerminalTrajectories** | Terminal command sequences and bash interactions | ~50% | | **OpenThoughts** | Chain-of-thought reasoning examples | ~50% | --- ## Dataset Characteristics - **Task types:** Shell commands, file operations, reasoning chains, structured outputs - **Style:** Technical, direct, focused on tool use and system interaction - **Quality:** Filtered for correctness, deduplicated with MinHash - **Augmentation:** Original examples with linguistic variations (3-5x) --- ## Training Configuration Used with the following hyperparameters: ```python # Stage 1 Training - Base model: Qwen/Qwen2.5-0.5B-Instruct - Batch size: 1 (micro), gradient_accumulation: 16 - Effective batch: 16 - Max sequence length: 1024 - Learning rate: 2e-5 - Optimizer: 8-bit AdamW - Epochs: 3 - Steps: ~2,058 - Duration: ~5.4 hours on AMD RX 7600 XT ``` --- ## Files - `train.jsonl` — Training examples (~10,989 after split) - `val.jsonl` — Validation examples (~5% split) - `README.md` — This documentation --- ## Usage ```python from datasets import load_dataset dataset = load_dataset("LumiVore/lumivore-stage1-training-data") train_data = dataset["train"] val_data = dataset["validation"] ``` --- ## Related - **Stage 2 Dataset:** `LumiVore/lumivore-stage2-training-data` — OpenClaw-specific fine-tuning - **Stage 3 Dataset:** `LumiVore/lumivore-stage3-identity-dataset` — Identity and conversational training - **Model:** `LumiVore/lumivore-1.2b` (when published) --- ## Citation If you use this dataset, please cite: ```bibtex @dataset{lumivore2026stage1, title={Lumivore Stage 1 Training Dataset}, author={LumiVore AI}, year={2026}, url={https://huggingface.co/datasets/LumiVore/lumivore-stage1-training-data} } ``` --- *Created for the Lumivore-1.2B training pipeline*