alibaba-pai/DistilQwen_100k

---
license: apache-2.0
dataset_info:
features:
- name: instruction
dtype: string
- name: output
dtype: string
splits:
- name: train
num_bytes: 231762824
num_examples: 100000
download_size: 123802051
dataset_size: 231762824
configs:
- config_name: default
data_files:
- split: train
path: data/train-*
---

To support community developers in avoiding the phenomenon of "catastrophic forgetting" when fine-tuning the DistilQwen2.5 model, we have open-sourced a portion of the dataset used for model training.

These datasets are designed to provide a solid foundation for model fine-tuning, helping to enhance the model's adaptability to new tasks while maintaining its performance on previous ones.

The released data covers various domains, including mathematics, coding, knowledge-based Q&A, instruction following, and creative generation, with a total volume of 10k samples.

When fine-tuning the model with their own data, users can incorporate DistilQwen_100k to ensure strong performance on downstream tasks without compromising the model's general capabilities, thereby preserving its generalization ability.

## Reference

For more detailed information about the dataset construction process, we encourage you to refer to our paper:

- **DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models**
Chengyu Wang, Junbing Yan, Yuanhao Yue, Jun Huang
[arXiv:2504.15027](https://arxiv.org/abs/2504.15027)

You can cite the paper using the following citation format:

```bibtex
@misc{wang2025distilqwen25industrialpracticestraining,
title={DistilQwen2.5: Industrial Practices of Training Distilled Open Lightweight Language Models},
author={Chengyu Wang and Junbing Yan and Yuanhao Yue and Jun Huang},
year={2025},
eprint={2504.15027},
archivePrefix={arXiv},
primaryClass={cs.CL},
url={https://arxiv.org/abs/2504.15027}
}
```

To support community developers in avoiding the phenomenon of catastrophic forgetting when fine-tuning the DistilQwen2.5 model, we have open-sourced a portion of the dataset used for model training. These datasets are designed to provide a solid foundation for model fine-tuning, helping to enhance the models adaptability to new tasks while maintaining its performance on previous ones. The released data covers various domains, including mathematics, coding, knowledge-based Q&A, instruction following, and creative generation, with a total volume of 10k samples.