remyxai/mhpd-dpo-v0
收藏资源简介：
MHPD-DPO v0.4 是一个用于论文推荐重排器DPO微调的成对偏好数据集，通过IPD框架从团队git合并历史中生成。该数据集是IPD方法的首次实例化（相关论文为《Code Is Context: Tuning Coding Agents via Implicit Preference Distillation》）。数据集包含4,418条成对偏好记录，覆盖两个Remyx项目：VQASynth（1,418对，每条记录前附有VQASynth的ExperimentHistory采样子集）和spacethinker（3,000对，每条记录前附有SpaceThinker模型卡的采样块，部分记录还包括VQASynth实验的“上游谱系”块，以反映SpaceThinker基于VQASynth生成数据训练的关系）。数据源自209篇独特的arXiv论文，共有2,242个独特的提示。DPO设置基于完成似然性，模型在推理时通过logprob(paper_text | prompt)计算分数以评估团队对齐程度。v0.4版本为两个兴趣项目提供了并行上下文块：VQASynth记录包含实验历史采样，spacethinker记录包含模型卡采样。数据集旨在通过trl.DPOTrainer和LoRA微调小型开放权重模型，作为对齐感知的重排器，在推理时注入目标项目的上下文以对候选论文进行排序。注意事项包括GP教师源基于VQASynth、单源训练限制、提示长度较长以及绝对效用值紧密聚类。
MHPD-DPO v0.4 is a pairwise preference dataset for DPO fine-tuning of paper-recommendation rerankers, generated from team git merge history via the IPD framework. It is the first instantiation of the broader IPD method (paper: *Code Is Context: Tuning Coding Agents via Implicit Preference Distillation*). The dataset contains 4,418 pairwise preference records across two Remyx projects: VQASynth (1,418 pairs, each prepended with a sampled subset of VQASynths ExperimentHistory) and spacethinker (3,000 pairs, each prepended with a sampled subset of the SpaceThinker model card, with some records including an upstream lineage block of VQASynth experiments, reflecting that SpaceThinker is trained on VQASynth-generated data). It is derived from 209 unique arXiv papers and includes 2,242 unique prompts. DPO setup operates at the completion-likelihood level, with inference using score = logprob(paper_text | prompt) to measure team alignment. Version v0.4 introduces parallel context blocks for both interests: VQASynth records feature sampled experiment history, while spacethinker records feature sampled model-card content. The dataset is intended for fine-tuning small open-weight models as alignment-aware rerankers via trl.DPOTrainer and LoRA, with inference involving injecting target project context into prompts to rank candidates. Caveats include the GP teacher source based on VQASynth, single-source-per-interest training, long prompt lengths, and tightly clustered absolute utility values.
