MagpieLM-SFT-Data-v0.1


Project Web: [https://magpie-align.github.io/](https://magpie-align.github.io/)

Arxiv Technical Report: [https://arxiv.org/abs/2406.08464](https://arxiv.org/abs/2406.08464)

Codes: [https://github.com/magpie-align/magpie](https://github.com/magpie-align/magpie)

## 🧐 Dataset Details
The Magpie Team generates this dataset for supervised fine-tuning. This dataset was used to train [Magpie-Align/MagpieLM-4B-SFT-v0.1](https://huggingface.co/Magpie-Align/MagpieLM-4B-SFT-v0.1).

To create this dataset, we first selected 550K high-quality Magpie **instructions** with diverse task categories (400K general + 150K reasoning), including
- 100K from [Magpie-Align/Magpie-Air-DPO-100K-v0.1](https://huggingface.co/datasets/Magpie-Align/Magpie-Air-DPO-100K-v0.1)
- 300K from [Magpie-Align/Magpie-Pro-MT-300K-v0.1](https://huggingface.co/datasets/Magpie-Align/Magpie-Pro-MT-300K-v0.1) (first turn only)
- 150K from [Magpie-Align/Magpie-Reasoning-150K](https://huggingface.co/datasets/Magpie-Align/Magpie-Reasoning-150K)

Then, we generate responses using [google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it).

Why Magpie 💜 Gemma-2-9B? Take a look at our latest paper: [Stronger Models are NOT Stronger Teachers for Instruction Tuning](https://huggingface.co/papers/2411.07133). We found that stronger models are not always stronger teachers for instruction tuning!

**License**: Please follow [Meta Llama 3.1 Community License](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE) and [Gemma License](https://www.kaggle.com/models/google/gemma/license/).

## 📚 Citation

If you find the model, data, or code useful, please cite our paper:
```
@article{xu2024magpie,
title={Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing},
author={Zhangchen Xu and Fengqing Jiang and Luyao Niu and Yuntian Deng and Radha Poovendran and Yejin Choi and Bill Yuchen Lin},
year={2024},
eprint={2406.08464},
archivePrefix={arXiv},
primaryClass={cs.CL}
}

@article{xu2024stronger,
title={Stronger Models are NOT Stronger Teachers for Instruction Tuning},
author={Xu, Zhangchen and Jiang, Fengqing and Niu, Luyao and Lin, Bill Yuchen and Poovendran, Radha},
journal={arXiv preprint arXiv:2411.07133},
year={2024}
}
```

**Contact**

Questions? Contact:
- [Zhangchen Xu](https://zhangchenxu.com/) [zxu9 at uw dot edu], and
- [Bill Yuchen Lin](https://yuchenlin.xyz/) [yuchenlin1995 at gmail dot com]


项目官网：[https://magpie-align.github.io/](https://magpie-align.github.io/)
ArXiv技术报告：[https://arxiv.org/abs/2406.08464](https://arxiv.org/abs/2406.08464)
代码仓库：[https://github.com/magpie-align/magpie](https://github.com/magpie-align/magpie)

## 🧐 数据集详情
喜鹊（Magpie）团队构建本数据集用于监督微调（Supervised Fine-Tuning, SFT），该数据集被用于训练[Magpie-Align/MagpieLM-4B-SFT-v0.1](https://huggingface.co/Magpie-Align/MagpieLM-4B-SFT-v0.1)。

为构建本数据集，我们首先筛选出55万条高质量的喜鹊（Magpie）**指令**，涵盖多样化的任务类别（40万通用任务 + 15万推理任务），具体来源包括：
- 10万条来自[Magpie-Align/Magpie-Air-DPO-100K-v0.1](https://huggingface.co/datasets/Magpie-Align/Magpie-Air-DPO-100K-v0.1)
- 30万条来自[Magpie-Align/Magpie-Pro-MT-300K-v0.1](https://huggingface.co/datasets/Magpie-Align/Magpie-Pro-MT-300K-v0.1)（仅取第一轮对话）
- 15万条来自[Magpie-Align/Magpie-Reasoning-150K](https://huggingface.co/datasets/Magpie-Align/Magpie-Reasoning-150K)

随后，我们使用[google/gemma-2-9b-it](https://huggingface.co/google/gemma-2-9b-it)生成对应回复。

为何选择喜鹊（Magpie）与Gemma-2-9B？不妨参阅我们的最新论文：[更强的模型并非指令微调的更佳教师](https://huggingface.co/papers/2411.07133)。我们的研究表明，更强的模型并不总能成为指令微调的更优教师！

**许可证**：请遵守[Meta Llama 3.1社区许可证](https://github.com/meta-llama/llama-models/blob/main/models/llama3_1/LICENSE)与[Gemma许可证](https://www.kaggle.com/models/google/gemma/license/)。

## 📚 引用
若您认为本模型、数据集或代码对您的研究有所帮助，请引用我们的论文：

@article{xu2024magpie,
title={Magpie: Alignment Data Synthesis from Scratch by Prompting Aligned LLMs with Nothing},
author={Zhangchen Xu and Fengqing Jiang and Luyao Niu and Yuntian Deng and Radha Poovendran and Yejin Choi and Bill Yuchen Lin},
year={2024},
eprint={2406.08464},
archivePrefix={arXiv},
primaryClass={cs.CL}
}

@article{xu2024stronger,
title={Stronger Models are NOT Stronger Teachers for Instruction Tuning},
author={Xu, Zhangchen and Jiang, Fengqing and Niu, Luyao and Lin, Bill Yuchen and Poovendran, Radha},
journal={arXiv preprint arXiv:2411.07133},
year={2024}
}

**联系方式**
如有疑问，请联系：
- [张晨旭（Zhangchen Xu）](https://zhangchenxu.com/) [e-mail]，以及
- [林宇辰（Bill Yuchen Lin）](https://yuchenlin.xyz/) [e-mail]