anttip/Tunesets_Edu_v2

Tunesets_Edu_v2是一个针对教育领域语言模型微调的高质量数据集混合体，主要关注非推理性的指令遵循任务，上下文大小大多在16k以下。该数据集的领域专注于非代码和非数学任务，并包含多语言数据。数据集通过对以下数据集进行过滤和采样得到：arcee-ai/The-Tome、microsoft/orca-agentinstruct-1M-v1等。处理流程包括基于规则的噪声和长度过滤、使用Minhash和字符串相似性的对话去重、基于LLM分类的过滤和平衡采样。数据集行经过Arcee AI微调的分类模型分类，并根据提供的提示进行合并分类。

Tunesets_Edu_v2 is a high-quality dataset blend for finetuning education-domain LLMs, focusing on non-reasoning instruction following tasks mostly under 16k context. The domain is concentrated on non-code and non-math tasks, including multi-lingual data. The dataset is created by filtering and sampling from various datasets like arcee-ai/The-Tome, microsoft/orca-agentinstruct-1M-v1, etc. The processing involves rule-based noise and length filtering, deduplication of conversations using Minhash and string similarities, and filtering and balanced sampling based on LLM classifications. The dataset rows are classified using Arcee AI finetuned models and merged based on the provided prompt.