allenai/olmo-2-1124-7b-preference-mix

OLMo 2 1124 7B Preference Mixture数据集是一个由多个策略偏好数据集混合而成的集合，主要用于DPO（Direct Preference Optimization）训练，以优化OLMo-2-1124-7B-SFT模型的性能。数据集包含366.7k生成对，涉及多个开源和商业模型，如Mistral、Tulu、Yi、MPT、Google Gemma、InternLM、Falcon、Qwen、GPT-4、Microsoft Phi和NuMind等。数据集遵循ODC-BY许可，适用于研究和教育用途。

This dataset is a mixture of preference data from multiple sources, used for Direct Preference Optimization (DPO). It includes prompts and feedback data generated from various sources and models, such as SFT mix, WildChat, Ultrafeedback, etc. The data in the dataset is generated by multiple models, including Mistral, Tulu, Yi, MPT, Google Gemma, InternLM, Falcon, Qwen, GPT-4, Microsoft Phi, and NuMind, among others. The dataset is licensed under ODC-BY, suitable for research and educational use, but some portions may be subject to other licensing terms.