driftbench/DriftBench

DriftBench是一个用于测量多轮LLM辅助科学构思中轨迹漂移的基准测试。该基准测试旨在评估模型在多次迭代压力下是否保持对结构化研究简报中硬约束的遵守。数据集包含研究简报、提示模板、评分标准、对话转录、法官评分、审计评分等多个组件，涵盖了24个科学领域的38个研究简报。基准测试设计了四种条件来测试多轮互动的不同方面，并使用了来自五个提供商的七种模型进行测试。关键发现包括所有模型在压力下都会增加复杂性，大多数模型会出现至少50%的漂移率，以及模型能够准确重述它们违反的约束。

DriftBench is a benchmark for measuring trajectory drift in multi-turn LLM-assisted scientific ideation. It evaluates whether models maintain hard constraints from a structured research brief across multiple turns of iterative pressure. The dataset includes research briefs, prompt templates, rubrics, conversation transcripts, judge scores, auditor scores, and more, covering 38 research briefs across 24 scientific domains. The benchmark tests four conditions to examine different aspects of multi-turn interaction and uses seven models from five providers. Key findings show that all models inflate complexity under pressure, most models drift at least 50%, and models accurately restate constraints they violate.