SL-AI/GRaPE-Base-Mix

---
license: apache-2.0
---

# GRaPE Base Mix

This dataset provides the initial training for GRaPE, taking a base model and making it conversational, and smart in many fields such as:

* Code

* Math

* Music Theory

* Instruction Following

* Tool Usage

* Long Context

There are a max token amount of 137,000 tokens.

There are 3M Examples from various datasets, all shuffled to ensure training goes smoothly.

### Datasets Used

#### General & Reasoning SFT
* **hermes-3**: General supervised fine-tuning
* **slimorca**: General SFT with a focus on reasoning
* **ultrafeedback**: General SFT derived from binarized feedback

#### Code & STEAM
* **glaive_code**: Code assistant SFT
* **evol_code**: Code instruction SFT
* **metamath**: Math reasoning SFT
* **music_theory**: Music theory SFT
* **music_lyrics**: Music lyrics SFT

#### Agentic & Function Calling
* **andy_4**: Agentic SFT with function calling for playing Minecraft
* **glaive_function_calling**: Function calling SFT
* **tool_ace**: Tool-use and agentic SFT
* **agentic_long_qa**: Agentic SFT for long-context question answering

#### Uncensored & Conditional SFT
* **toxic_qa_sft**: Uncensored SFT
* **nitral_ai**: Uncensored SFT
* **toxic_dpo1**: DPO dataset converted to conditional SFT
* **toxic_dpo2**: DPO dataset converted to conditional SFT

*(Note that these datasets were used to train the model when it is and when it is not appropriate to be uncensored.)*