mkd-chanwoo/keural-DPO

keural-DPO是一个用于韩语/英语直接偏好优化（DPO）训练的统一数据集。它整合了六个来源的数据集，包括ultrafeedback_binarized、multifaceted_collection_dpo、hh_rlhf、orca_dpo_pairs_ko、aihub_71748和aihub_71760，并将它们规范化为ChatML格式。数据集总共包含443,314个样本，其中语言分布为：仅英语占49.7%（220,499个样本），仅韩语占35.8%（158,721个样本），英语/韩语混合占14.5%（64,094个样本）。数据集涵盖多种偏好信号类型，如帮助性、无害性/安全性、诚实性/真实性、风格/角色和响应质量排名，强调“看似合理但不够适当的答案 vs 更好的答案”这种微妙偏好对，以提高DPO信号质量。数据格式包括prompt（第一个用户问题）、chosen_chatml（选择的响应ChatML序列）和rejected_chatml（拒绝的响应ChatML序列），支持单轮和多轮对话。数据集适用于文本生成任务，特别用于DPO、对齐和RLHF相关研究，并需在训练时添加ChatML特殊令牌。许可证遵循各来源数据集的规定，包括MIT、Apache-2.0和AI Hub利用条款。

keural-DPO is a unified dataset for Korean/English Direct Preference Optimization (DPO) training. It integrates six source datasets—ultrafeedback_binarized, multifaceted_collection_dpo, hh_rlhf, orca_dpo_pairs_ko, aihub_71748, and aihub_71760—normalized into ChatML format. The dataset contains a total of 443,314 samples, with language distribution: English only at 49.7% (220,499 samples), Korean only at 35.8% (158,721 samples), and English/Korean mixed at 14.5% (64,094 samples). It covers various preference signal types, such as helpfulness, harmlessness/safety, honesty/truthfulness, style/persona, and response quality ranking, emphasizing subtle preference pairs like plausible but less appropriate answers vs better answers to enhance DPO signal quality. The data format includes prompt (the first user question), chosen_chatml (the ChatML sequence for the chosen response), and rejected_chatml (the ChatML sequence for the rejected response), supporting both single-turn and multi-turn dialogues. It is designed for text-generation tasks, particularly for DPO, alignment, and RLHF-related research, and requires adding ChatML special tokens during training. Licenses follow the original source datasets, including MIT, Apache-2.0, and AI Hub terms of use.